Most Science Studies Appear to Be Tainted By Sloppy Analysis

We all make mistakes and, if you believe medical scholar John Ioannidis, scientists make more than their fair share. By his calculations, most published research findings are wrong. Dr. Ioannidis is an epidemiologist who studies research methods at the University of Ioannina School of Medicine in Greece and Tufts University in Medford, Mass. In a series of influential analytical reports, he has documented how, in thousands of peer-reviewed research papers published every year, there may be so much less than meets the eye.

These flawed findings, for the most part, stem not from fraud or formal misconduct, but from more mundane misbehavior: miscalculation, poor study design or self-serving data analysis. "There is an increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims," Dr. Ioannidis said. "A new claim about a research finding is more likely to be false than true."

The hotter the field of research the more likely its published findings should be viewed skeptically, he determined. Take the discovery that the risk of disease may vary between men and women, depending on their genes. Studies have prominently reported such sex differences for hypertension, schizophrenia and multiple sclerosis, as well as lung cancer and heart attacks. In research published last month in the Journal of the American Medical Association, Dr. Ioannidis and his colleagues analyzed 432 published research claims concerning gender and genes. Upon closer scrutiny, almost none of them held up. Only one was replicated.

Statistically speaking, science suffers from an excess of significance. Overeager researchers often tinker too much with the statistical variables of their analysis to coax any meaningful insight from their data sets. "People are messing around with the data to find anything that seems significant, to show they have found something that is new and unusual," Dr. Ioannidis said.

In the U. S., research is a $55-billion-a-year enterprise that stakes its credibility on the reliability of evidence and the work of Dr. Ioannidis strikes a raw nerve. In fact, his 2005 essay "Why Most Published Research Findings Are False" remains the most downloaded technical paper that the journal PLoS Medicine has ever published.

"He has done systematic looks at the published literature and empirically shown us what we know deep inside our hearts," said Muin Khoury, director of the National Office of Public Health Genomics at the U.S. Centers for Disease Control and Prevention. "We need to pay more attention to the replication of published scientific results."

Every new fact discovered through experiment represents a foothold in the unknown. In a wilderness of knowledge, it can be difficult to distinguish error from fraud, sloppiness from deception, eagerness from greed or, increasingly, scientific conviction from partisan passion. As scientific findings become fodder for political policy wars over matters from stem-cell research to global warming, even trivial errors and corrections can have larger consequences.

Still, other researchers warn not to fear all mistakes. Error is as much a part of science as discovery. It is the inevitable byproduct of a search for truth that must proceed by trial and error. "Where you have new areas of knowledge developing, then the science is going to be disputed, subject to errors arising from inadequate data or the failure to recognize new matters," said Yale University science historian Daniel Kevles. Conflicting data and differences of interpretation are common.

To root out mistakes, scientists rely on each other to be vigilant. Even so, findings too rarely are checked by others or independently replicated. Retractions, while more common, are still relatively infrequent. Findings that have been refuted can linger in the scientific literature for years to be cited unwittingly by other researchers, compounding the errors.

Stung by frauds in physics, biology and medicine, research journals recently adopted more stringent safeguards to protect at least against deliberate fabrication of data. But it is hard to admit even honest error. Last month, the Chinese government proposed a new law to allow its scientists to admit failures without penalty. Next week, the first world conference on research integrity convenes in Lisbon.

Overall, technical reviewers are hard-pressed to detect every anomaly. On average, researchers submit about 12,000 papers annually just to the weekly peer-reviewed journal Science. Last year, four papers in Science were retracted. A dozen others were corrected.

No one actually knows how many incorrect research reports remain unchallenged. Earlier this year, informatics expert Murat Cokol and his colleagues at Columbia University sorted through 9.4 million research papers at the U.S. National Library of Medicine published from 1950 through 2004 in 4,000 journals. By raw count, just 596 had been formally retracted, Dr. Cokol reported. "The correction isn't the ultimate truth either," Prof. Kevles said.

Source





Daily pill may save blood clot patients

A pill taken once a day could save the lives of thousands of the people who die in English hospitals every year from blood clots, according to a study published in The Lancet medical journal. Venous thrombo-embolism (VTE) - or blood clot - kills 25,000 people a year in English hospitals, more than the number of people who die from breast cancer, Aids or road traffic accidents.

The current treatment used in hospitals to prevent blood clots after hip replacement surgery is the injection Heparin. Yet a number of elderly patients, often discharged a few days after surgery, could find it difficult to inject themselves or to return to hospital for injections. Data has shown that the pill Dabigatran, taken once a day for an average of 33 days, is as effective as Heparin in preventing blood clots after hip replacement surgery. It will be available early next year and will be the first medicine available as a pill to prevent blood clots in hospital.

Patients undergoing surgery are particularly at risk of getting blood clots - a third of patients who are readmitted are done so because of deep vein thrombosis. Evidence shows that it takes an average of 21 days for a patient who has undergone a total hip replacement to develop a VTE, and that in three quarters of cases this was after they had been discharged from hospital. Guidelines from the Department of Health and the National Institute for Health and Clinical Excellence (Nice), which highlighted the problem, say that at-risk patients should receive treatment for a month following hip replacement surgery. A pill could therefore be more convenient.

Prof Simon Frostick, professor of orthopaedics at the University of Liverpool, who conducted the study, said: "Given the trend for shorter hospital stays following joint replacement surgery, it is becoming increasingly important to have anti-coagulant treatments available which are safe and easy to use. "Once-daily oral Dabigatran may be an attractive alternative to other regimens." This comes after Sir Liam Donaldson, the chief medical officer for England, said there was "room for improvement" in the treatment of patients at risk of blood clots. He said in a foreword to a Department of Health report in March: "The evidence suggested that in England around 25,000 people a year died from VTE in hospitals alone."

Source

****************

Just some problems with the "Obesity" war:

1). It tries to impose behavior change on everybody -- when most of those targeted are not obese and hence have no reason to change their behaviour. It is a form of punishing the innocent and the guilty alike. (It is also typical of Leftist thinking: Scorning the individual and capable of dealing with large groups only).

2). The longevity research all leads to the conclusion that it is people of MIDDLING weight who live longest -- not slim people. So the "epidemic" of obesity is in fact largely an "epidemic" of living longer.

3). It is total calorie intake that makes you fat -- not where you get your calories. Policies that attack only the source of the calories (e.g. "junk food") without addressing total calorie intake are hence pissing into the wind. People involuntarily deprived of their preferred calorie intake from one source are highly likely to seek and find their calories elsewhere.

4). So-called junk food is perfectly nutritious. A big Mac meal comprises meat, bread, salad and potatoes -- which is a mainstream Western diet. If that is bad then we are all in big trouble.

5). Food warriors demonize salt and fat. But we need a daily salt intake to counter salt-loss through perspiration and the research shows that people on salt-restricted diets die SOONER. And Eskimos eat huge amounts of fat with no apparent ill-effects. And the average home-cooked roast dinner has LOTS of fat. Will we ban roast dinners?

6). The foods restricted are often no more calorific than those permitted -- such as milk and fruit-juice drinks.

7). Tendency to weight is mostly genetic and is therefore not readily susceptible to voluntary behaviour change.

8). And when are we going to ban cheese? Cheese is a concentrated calorie bomb and has lots of that wicked animal fat in it too. Wouldn't we all be better off without it? And what about butter and margarine? They are just about pure fat. Surely they should be treated as contraband in kids' lunchboxes! [/sarcasm].

Trans fats:

For one summary of the weak science behind the "trans-fat" hysteria, see here. Trans fats have only a temporary effect on blood chemistry and the evidence of lasting harm from them is dubious. By taking extreme groups in trans fats intake, some weak association with coronary heart disease has at times been shown in some sub-populations but extreme group studies are inherently at risk of confounding with other factors and are intrinsically of little interest to the average person.

*********************