On the morning of March 2, 2005, a 14-year-old Japanese girl woke up scared. At first she thought someone was outside the house watching her, but then she decided the stranger must be inside. She wandered restlessly and, despite the cold weather, threw open all the windows. Later, over a meal, she declared, "The salad is poisoned." Two days later, she said she wanted to kill herself.
This teenager with no history of mental illness was diagnosed with delirium. The night before the hallucinations started, she began taking an anti-influenza drug called Tamiflu (generic name: oseltamivir), which governments around the world have spent billions stockpiling for the next major flu outbreak.
But evidence released earlier this year by Cochrane Collaboration, a London-based nonprofit, shows that a significant amount of negative data from the drug's clinical trials were hidden from the public. The Food and Drug Administration (FDA) knew about it, but the medical community did not; the U.S. Centers for Disease Control and Prevention (CDC), which doesn't have the same access to unpublished data as regulators, had recommended the drug without being able to see the full picture. When results from those unpublished trials finally did emerge, they cast doubt over whether Tamiflu is as effective as the manufacturer says.
The revelation of hidden data bolstered a growing movement against what's referred to within the research community as "publication bias," in which scientists squirrel away mostly negative or inconclusive findings and broadcast only their positive ones. Concealing trial data—for which patients accept the risks of untested treatments for the greater good—is routine. As many as half of all clinical trials are never published, PLOS Medicine reported last year.
That Japanese girl, whose case was detailed in an FDA report, did not kill herself. But at least 70 people have died, many of them by suicide, after Tamiflu-induced episodes. The deaths were almost surreal: A 14-year-old who took Tamiflu jumped off a balcony, and a 17-year-old on the drug ran in front of a truck. Scientists documented other cases of "psychopathic events," including a South Korean girl who temporarily developed bipolar disorder and an 8-year-old Japanese boy who wouldn't answer to his name and began to growl.
Tens of millions of people have taken Tamiflu without incident, and you are far more likely to die from the flu than you are to have a dangerous reaction to the drug. And it's true that with many medicines a minuscule chance of death is usually tolerated—as long as the benefits far outweigh that risk.
But if Tamiflu does nothing, and there's even a slight chance of life-threatening side effects, why was it approved? And why continue to prescribe it? That's what the Cochrane Collaboration argued in a report it published in April. Cochrane is widely considered one of the most rigorous reviewers of health science data. It takes results of multiple trials, looks for faults and draws conclusions. It doesn't accept funding from businesses with a stake in its findings. And in April, Cochrane took Tamiflu's maker, Roche, to task about many of its claims.
Questionable Findings
The influenza virus infects by hijacking healthy cells in the respiratory tract, taking over the nucleus—the cell's brain—and ordering up copies of itself. Then these new viruses burst out of the cell membrane like an enemy horde to infect more cells. The drug oseltamivir is thought to work mainly by trapping those virus copies inside the cell. With a lower "viral load," the symptoms won't last as long, and there's a lesser chance of developing complications, like pneumonia, and infecting other people.
That was the theory. Roche paid for dozens of clinical trials to prove oseltamivir worked in practice. Afterward, the scientists produced lengthy "clinical study reports" and turned them over to the FDA, which approved the drug in 1999. Some of those reports were condensed into short articles published in medical journals. Most of us have access only to these published findings.
But the full clinical study reports from those trials were locked away. There wasn't even a list of what trials were conducted; Cochrane initially figured there were about 36 and sought to read them all. It began negotiating with the drugmaker, filing freedom of information requests to the FDA and the European Medicines Agency (EMA), Europe's FDA equivalent, and backing pressure campaigns in the media. At one point early on, Roche offered to give Cochrane 10 reports, but only if it signed a confidentiality agreement keeping everything secret—including the existence of the agreement. Cochrane refused to sign.
After five years, Roche and the EMA opened up (the FDA has not, and did not respond to requests, both from Cochrane and Newsweek, to explain why). What Cochrane uncovered was more than 70 Tamiflu trials and well over 100,000 pages of unpublished reports. Among them were many trials where the results were negative or inconclusive. With this more complete picture of the testing, Cochrane concluded the trials don't prove that Tamiflu prevents hospitalizations, contagiousness or complications. The only thing it definitely does do, Cochrane said, is shorten the duration of symptoms, by about a day.
Many agencies, including the CDC, did not change their positions after the report. In response to questions from Newsweek, a CDC spokeswoman referred to an article from April in which the agency says it still believes Tamiflu is "an important adjunct to influenza vaccine." It cites, for example, a large study published in May in The Lancet, which found that the odds of dying from the flu rose each day without Tamiflu.
As the Cochrane Collaboration was conducting its investigation, the BMJ was conducting its own. It found that neither the FDA nor the EMA "logged any specific psychiatric harms when they licensed oseltamivir." No one could have predicted the bizarre suicides, but Cochrane's disclosures revealed indications in the unpublished study reports of the potential for psychiatric side effects. Roche and the regulatory agencies have said there was no proof that the delirium and hallucinations were caused by the drug. As the number of deaths mounted, that position became increasingly difficult to hold.
Since the Cochrane report, many groups have urged companies and regulators to open up trial data, particularly if public money is being used. The U.S. government spent $1.3 billion to develop and stockpile antiviral medication, including Tamiflu. "All trials should be published or at least have their results available somewhere," says Agnes Dechartres of Paris Descartes University, who led the PLOS Medicine study.
Bad Science
The consequences of exclusion or delay of trial data have ranged from frustration to mass fatalities. When one doctor in Italy was diagnosed with bone cancer, he wanted to know whether a stem cell transplant would offer hope of a cure. Four clinical trials had been conducted, but none had been fully published. "Why was I forced to make my decision knowing that information was somewhere but not available?" he wrote in the BMJ. "Was the delay because the results were less exciting than expected?"
The most infamous case of publication bias is a 1980 study in which heart attack patients were split into two groups: One group received a drug called lorcainide, while the other group received a placebo. Researchers wanted to find out whether lorcainide cures dangerous abnormal heart rhythms, which it does. But during the study, nine of the lorcainide patients died, compared with just one of the placebo patients. Lorcainide's maker stopped production soon after for unrelated commercial reasons, and consequently the researchers never published their report. They believed the deaths to be "an effect of chance."
But physicians continued prescribing drugs similar to lorcainide. Finally, in 1993, the researchers apologized for withholding the study, which they said could have "provided an early warning of trouble ahead." But the damage was done: Throughout the '80s, an estimated 20,000 to 75,000 people died in the U.S. each year from the inappropriate prescription of antiarrhythmic drugs, according to a 2003 editorial by Kay Dickersin and Drummond Rennie in The Journal of the American Medical Association.
Funders, researchers and journal editors who do not publish otherwise publishable findings are breaking ethical and legal rules. The Declaration of Helsinki, which establishes global ethics guidelines for medical testing on humans, states that all results must be made public. In the U.S., failing to register clinical trial results has been against the law since 2007. Because Tamiflu was approved in 1999, Roche has never been penalized. Nevertheless, even with the requirements in effect, results are still often left unregistered.
Sometimes the bias is more subtle. Results get published, but negative details are left out.
In 2001, Ellen Roche, a 24-year-old technician at Johns Hopkins University, volunteered for a study to find out how healthy people's bodies avoid asthma attacks. Doctors would give her a dose of the drug hexamethonium to induce a mild attack, her healthy body would fend off that attack, and doctors would observe how her body did it. Instead, she began to cough, her lung tissue broke down, and her kidneys failed. She was dead within a month. A study performed way back in 1978 had discovered this kind of reaction to hexamethonium—but the paper that was published didn't mention it.
Knowing about negative results is not just useful; it is essential to good science. Randomized clinical trials are considered the best way to test a drug: Get two groups of patients with the same problem, give one group an experimental treatment and see if it works better than no treatment. Even better, amass the results of dozens of similar clinical trials, level the differences and draw better conclusions from a larger amount of data. That's called a systematic review.
Problem is, a systematic review only works if the reviewers have all the information—good, bad and inconclusive.
"In baseball, it is easy to find out just how well Cal Ripken has hit against various pitchers in the past, at home or away games, in recent weeks or during his career," Dickersin and Rennie wrote. "Yet in medicine, there is no comprehensive source for finding out similar, accurate statistics for medical interventions. How can baseball be better organized and keep better records than medical science?"
Drugmaker Influence
One reason is that drug companies have so much to lose. "The potential economic implications of publishing an unfavorable study are really significant for pharmaceutical companies," says Christopher W. Jones, who studies publication bias at Cooper Medical School of Rowan University. "And clearly those incentives have an effect on the way they choose to present their data to the public."
Last year, Jones looked at 585 major clinical trials set to begin prior to 2009. By November 2012, nearly a third—including about a quarter-million test patients—remained unpublished, Jones and his co-authors reported in a BMJ article. Industry-funded trial results were more likely (32 percent) to be unpublished than those without industry funding (18 percent). (Other studies, including the one by Dechartres, showed the proportion is about the same.)
Worse, even systematic reviews, which help set standard medical practices, are subject to the influence of drug companies. An October 7 paper in Annals of Internal Medicine found that scientists hired by the manufacturer to assess the efficacy of a drug were more likely to paint a prettier picture than independent reviewers.
The subject of that study? Anti-viral flu medications, including Tamiflu.
Roche, through its subsidiary, Genentech, did not directly answer the findings of that paper but said in statements to Newsweek that the company "regularly works alongside established experts" and that "Genentech's professional relationships with experts are within all existing pharmaceutical industry and regulatory guidelines."
Although Roche "fundamentally disagrees" with Cochrane's findings, lately the company has begun to distance itself from the rest of the industry, even supporting open trials measures in Europe. Pharmaceutical Research and Manufacturers of America has generally opposed the open trials movement, arguing it risks exposing trade secrets. But Austine Graff, a Genentech spokeswoman, says that "greater access to patient-level data"—the rawest form of data—"helps ensure the data provided by research participants are used to maximum effect in the creation of knowledge and understanding."
Asked why Roche initially resisted Cochrane's data requests so strongly, Graff indicated the company was caught off guard. She said Roche's reaction was "in line with industry standards at the time," and it was "not sure if we could legally share all the individual data with them due to patient confidentiality."
Publication bias in clinical trials was, for a long time, something only scientists cared about. But lately the issue has raised a crop of lobbyists and gone mainstream. In 2005, John P.A. Ioannidis, a researcher at the Stanford School of Medicine, published a scientific paper under the provocative heading "Why Most Published Research Findings Are False" and mentioned publication bias. And British doctor Ben Goldacre leads a movement called AllTrials. His TED talk on publication bias has been seen 1.8 million times.
Various organizations and national health agencies are working on solutions. They've created online registries where researchers announce that they're starting a trial and then post the results when they're done. But they frequently do neither. So now, the World Health Organization (WHO) is preparing an official statement saying that "there is an ethical imperative" to use these registries.
It is uncertain if the WHO's "imperative" will carry more force than the Declaration of Helsinki or even U.S. law. But without more serious personal consequences for violations, it's not likely.
Baseball: 2. Medicine: 0.
