Attractive myths

It long ago became clear to me from my own research and its reception that the popularity and attractiveness of a theory had much more to do with acceptance of it than the evidence for it. See e.g. here. Indeed, the evidence seemed to be quite irrelevant if a particular theory was very attractive to most people in the field. My observations were in psychology but the theory that a low fat diet is beneficial is a close equivalent in medicine. So I am pleased to read now that one group of researchers thinks headline-grabbing scientific reports are the most likely to turn out to be wrong

IN ECONOMIC theory the winner’s curse refers to the idea that someone who places the winning bid in an auction may have paid too much. Consider, for example, bids to develop an oil field. Most of the offers are likely to cluster around the true value of the resource, so the highest bidder probably paid too much.

The same thing may be happening in scientific publishing, according to a new analysis. With so many scientific papers chasing so few pages in the most prestigious journals, the winners could be the ones most likely to oversell themselves—to trumpet dramatic or important results that later turn out to be false. This would produce a distorted picture of scientific knowledge, with less dramatic (but more accurate) results either relegated to obscure journals or left unpublished.

In Public Library of Science (PloS) Medicine, an online journal, John Ioannidis, an epidemiologist at Ioannina School of Medicine, Greece, and his colleagues, suggest that a variety of economic conditions, such as oligopolies, artificial scarcities and the winner’s curse, may have analogies in scientific publishing.

Dr Ioannidis made a splash three years ago by arguing, quite convincingly, that most published scientific research is wrong. Now, along with Neal Young of the National Institutes of Health in Maryland and Omar Al-Ubaydli, an economist at George Mason University in Fairfax, Virginia, he suggests why.

It starts with the nuts and bolts of scientific publishing. Hundreds of thousands of scientific researchers are hired, promoted and funded according not only to how much work they produce, but also to where it gets published. For many, the ultimate accolade is to appear in a journal like Nature or Science. Such publications boast that they are very selective, turning down the vast majority of papers that are submitted to them.

The assumption is that, as a result, such journals publish only the best scientific work. But Dr Ioannidis and his colleagues argue that the reputations of the journals are pumped up by an artificial scarcity of the kind that keeps diamonds expensive. And such a scarcity, they suggest, can make it more likely that the leading journals will publish dramatic, but what may ultimately turn out to be incorrect, research.

Dr Ioannidis based his earlier argument about incorrect research partly on a study of 49 papers in leading journals that had been cited by more than 1,000 other scientists. They were, in other words, well-regarded research. But he found that, within only a few years, almost a third of the papers had been refuted by other studies. For the idea of the winner’s curse to hold, papers published in less-well-known journals should be more reliable; but that has not yet been established.

The group’s more general argument is that scientific research is so difficult—the sample sizes must be big and the analysis rigorous—that most research may end up being wrong. And the “hotter” the field, the greater the competition is and the more likely it is that published research in top journals could be wrong.

There also seems to be a bias towards publishing positive results. For instance, a study earlier this year found that among the studies submitted to America’s Food and Drug Administration about the effectiveness of antidepressants, almost all of those with positive results were published, whereas very few of those with negative results were. But negative results are potentially just as informative as positive results, if not as exciting.

The researchers are not suggesting fraud, just that the way scientific publishing works makes it more likely that incorrect findings end up in print. They suggest that, as the marginal cost of publishing a lot more material is minimal on the internet, all research that meets a certain quality threshold should be published online. Preference might even be given to studies that show negative results or those with the highest quality of study methods and interpretation, regardless of the results.

It seems likely that the danger of a winner’s curse does exist in scientific publishing. Yet it may also be that editors and referees are aware of this risk, and succeed in counteracting it. Even if they do not, with a world awash in new science the prestigious journals provide an informed filter. The question for Dr Ioannidis is that now his latest work has been accepted by a journal, is that reason to doubt it?

Source





Sugar could keep prejudices under control??

Pretty absurd. Almost certainly a random result. We await replication. Note that stereotyping has long ago been shown to be strongly influenced by reality. So the results could be interpreted as showing that people high on sugar had less access to their memories

A spoonful of sugar could help the bigotry go down. Sweet drinks can have an effect in keeping prejudices under control, researchers claim. The reason, they believe, is that a sugar rush provides the brain with the fuel it needs to suppress unpleasant opinions. 'Ensuring people have sufficient energy for self-control may help to improve their ability to control both prejudice and use of stereotypes in their day-to-day life,' they say.

The anti-Alf Garnett effect of sucrose was discovered by giving a group of students lemonade. Some was sweetened with sugar, the rest a calorie-free version with an artificial sweetener. The men and women were then shown a picture of Sammy, a young man who was said to be gay, and asked to write for five minutes about what he did during a typical day.

The researchers counted how many times each person used words about Sammy from a list of 58 traits or characteristics associated with the stereotype of gay men, including feminine and artistic. Results show that those who had the sugary drink used far fewer stereotypes in their essays than those who had the artificial sweetener. The study, published in the Journal of Experimental Social Psychology, also showed that sugar's effect was greatest in the most prejudiced. 'The findings suggest a link between glucose levels and the expression of prejudice and the use of stereotypes,' it said. 'People with lower glucose levels are more likely to use stereotypes when describing others and, if they are high in prejudice, are more likely to make derogatory statements.'

The researchers, from Amsterdam University and Florida State University, believe the glucose derived from sugar provides the brain with the energy needed to keep objectionable thoughts to ourselves. The theory is based on the idea that most people, even those with high levels of prejudice, exercise some level of control over public expressions of their views.

The researchers said: 'When people engage in the act of trying to control public expressions of prejudice or the use of stereotypes, they consume the energy required for self-regulation. 'However, once the energy source is restored to normal levels, people regain the ability to control conscious responses towards others. 'Because self-control depends on processes that consume glucose as an energy source, people who have lower levels of blood glucose may be more likely to express prejudice.'

Source