Two years ago, Ars published a story about some famous psychology research that smelled… off. Psychologist Nicolas Guéguen’s flashy findings on human sexuality appeared to be riddled with errors and inconsistencies, and two researchers had raised an alarm.

Now, four years after James Heathers and Nick Brown first started digging into Guéguen’s work, one of his papers has been retracted. The study reported that men were more helpful to women wearing high heels compared to mid heels or flats. “As a man I can see that I prefer to see my wife when she wears high heels, and many men in France have the same evaluation,” Guéguen told Time in its coverage of the paper.

Slow progress

Since Brown and Heathers went public with their critiques of Guéguen’s work, there has been little progress. In September 2018, a meeting between Guéguen and university authorities concluded with an agreement that he would request retractions of two of his articles. One of those papers is the recently retracted high-heels study; the other was a study reporting that men prefer to pick up female hitchhikers who were wearing red compared to other colors. The latter has not yet been retracted.

In this meeting, Guéguen admitted to basing his publications on results from undergraduate fieldwork, without crediting the students. Nick Brown reports on his blog that he has been contacted by an anonymous student of Guéguen’s who claims that the undergraduate students in Guéguen’s course knew nothing about statistics and that “many students simply invented their data” for their fieldwork projects. The student provided an undergraduate field study report that is similar to Guéguen’s 2015 paper on men’s preference for helping women who wear their hair loose. The report appears to include some of the statistically improbable data that appeared in the paper.

It is not clear what the outcome has been of any university investigations. As recently as last month, French publication Le Télégramme reported that Guéguen was running for the position of dean of his faculty and lost the election after receiving nine out of 23 votes.

Black-box workings

The retraction notice for the high-heels paper reports that it was retracted at the request of the University of Southern Brittany, Guéguen’s institution.

“Following an institutional investigation, it was concluded that the article has serious methodological weaknesses and statistical errors,” states the retraction notice. “The author has not responded to any correspondence about this retraction.”

No further information is available about precisely what statistical errors led to the retraction. Brown and Heathers had identified a range of concerns, including some odd reporting of the sample sizes.

The experimenters tested people’s helpfulness based on their shoe height and were instructed to test 10 men and 10 women before changing their shoes. With three different shoe heights, this should have meant 60 participants for each experimenter, or even 80, 100, or 120 if they repeated a shoe height. Yet the paper reports instead a sample size that works out to 90 participants per experimenter. That makes it unclear how many people were tested with each shoe height and by each experimenter and, more generally, how accurately the experiment was reported in the paper. Brown and Heathers also found some errors in the statistical tests, in which the results didn’t match up with the data reported in the paper.

Because the retraction notice is vague, the high-heels paper could have been retracted based on these concerns. But other problems could also have been identified. “It’s actually quite uncommon for an explicit retraction notice to explain what went wrong and how it worked,” Heathers told Ars. Most of the time, he says, “it goes into a system and there’s a black box outcome at the end.”

In June this year, the editors of the International Review of Social Psychology published an “expression of concern” about six of Guéguen’s papers that had been published in their journal. They had requested an investigation of Guéguen’s work and agreed to follow the recommendations of the investigator. Despite the investigator recommending a retraction of two of Guéguen’s papers in other journals, the editors decided instead to opt for an expression of concern.

“The report concludes misconduct,” the editors write. “However, the standards for conducting and evaluating research have evolved since [Guéguen] published these articles, and for that reason, we instead believe it is difficult to establish with sufficient certainty that scientific misconduct has occurred.”

Brown and Heathers critiqued 10 of Guéguen’s papers. So far, this paper is the first to have been retracted.

Media coverage

When the high-heels paper was published, it attracted an avalanche of media attention. Brown has tweeted at 30 journalists and bloggers who covered the study, asking them if they will be correcting their original pieces. He didn’t expect anything to come of it, he told Ars; it was more an expression of outrage.

Finding out down the line that a paper has been retracted is an occupational hazard of science news. Reasons for retraction run the gamut from outright fraud to unintentional errors that the researchers are mortified to discover. Other retractions seem largely out of their control. In some cases, the researchers themselves are the ones who report the errors and request the retraction.

Obviously it’s important to screen the quality of the research you’re covering, but for science reporters, the only way to be completely sure that you’ll never cover work that could be retracted is to never cover anything at all.

That said, how reporters respond to retractions matters. One concern is that this coverage will probably remain unaltered in the majority of outlets, where it can be linked to and used as a source—readers will have no indication that the research it covers is highly questionable. Ars has historically posted a note in the article and altered the headline when we become aware that work we have covered has been retracted. But we’ll now be adding to that policy by committing to also posting a short piece about the retraction and explain the reasons behind it if possible. Since retractions often don’t receive much fanfare, they can be easy to miss, so please contact us if you’re aware of retractions for any research that we’ve covered.

https://arstechnica.com/?p=1586533