A new review of the influenza drug oseltamivir (Tamiflu) has raised questions about both the efficacy of the medication and the commitment of its maker to supply enough data for claims about the drug to be evaluated by independent experts.
It also raises questions about the entire process of systematic review.
Researchers led by Tom Jefferson, MD, of the Cochrane Collaboration, pored over 15 published studies and nearly 30,000 pages of “clinical study reports.”
But, they reported, the clinical study information – data previously shared only with regulators – was only a part of what internal evidence suggested was available.
And many published studies had to be excluded because of missing or contradictory data, Jefferson and colleagues reported.
Activate MedPage Today’s CME feature and receive free CME credit on medical stories like this one
Action Points
- Explain that a new review of an important flu drug has raised questions about the medication and the entire process of systematic review.
- Point out that the review of oseltamivir showed that there was no evidence of effect on hospital admissions.
The drug’s maker, Switzerland-based Roche, had promised after a previous Cochrane review to make all of its data available for “legitimate analyses.” After a request for the data, Jefferson and colleagues reported, the company sent them 3,195 pages covering 10 treatment trials of the drug.
But, three of the reviewers noted in a parallel report in BMJ, the tables of contents suggested that the data were incomplete.
“What we’re seeing is largely Chapter One and Chapter Two of reports that usually have four or five chapters,” according to theBMJ article’s lead author, Peter Doshi, PhD, of Johns Hopkins University.
Roche did not immediately respond to a telephoned request for comment.
Requests for More Data
The researchers then asked the European Medicines Agency (EMA) for the data, under a Freedom of Information request, and obtained a further 25,453 pages, covering 19 trials.
But that data, too, was incomplete, they said, although the agency said it was all that was available.
The FDA is thought to have the complete reports, but has not yet responded to requests for them, the researchers reported.
Regulatory agencies such as the EMA and FDA routinely see the large clinical study reports, Jefferson and colleagues said in BMJ, but systematic reviewers and the general medical public do not.
“While regulators and systematic reviewers may assess the same clinical trials, the data they look at differs substantially,” they said.
The Cochrane group has been trying for several years to put together a clear-cut systematic review of the evidence on antivirals aimed at flu.
In 2006, the group concluded that the evidence showed that oseltamivir reduced the complications of the flu. But that conclusion was challenged on the basis that a key piece of data was flawed.
An updated review in 2009 – throwing out the flawed study — concluded there wasn’t enough evidence to show that the drug had any effect on complications.
For this analysis, the Cochrane reviewers had originally intended to perform a systematic review on both of the approved neuraminidase inhibitors – oseltamivir and zanamivir (Relenza), using the clinical study reports to supplement published trials.
In the end, they decided that for oseltamivir, they needed more detail in order to perform the review in its entirety. But, they reported, some conclusions could be drawn from published data on the 15 trials and from 16,000 pages of clinical study reports that were available before their deadline.
They also decided to postpone analysis of zanamivir (for which they had 10 trials) because the drug’s maker, GlaxoSmithKline, offered individual patient data which they wanted time to analyze.
The oseltamivir analysis showed:
- The time to first alleviation of symptoms in people with influenza-like illness was a median of 160 hours in the placebo groups and about 21 hours shorter in those treated with oseltamivir. The difference, evaluated in five studies, was significant at P<0.001.
- There was no evidence of effect on hospital admissions: In seven studies, the odds ratio was 0.95, with a 95% confidence interval from 0.57 to 1.61, which was nonsignificant atP=0.86.
- A post-protocol analysis of eight studies showed that oseltamivir patients were less likely to be diagnosed with influenza.
- The data “lacked sufficient detail to credibly assess” any effect on influenza complications and viral transmission.
Data Discrepancies Found
But discrepancies between the published trial data and the clinical study reports “led us to lose confidence in the journal reports,” Doshi and colleagues wrote in BMJ.
For example, they noted that one journal report clearly said there were no drug-related serious adverse events, but the clinical study report listed three that were possibly related to oseltamivir.
As well, the sheer scope of the clinical study reports meant that much was left out of journal reports. One 2010 study, on safety and pharmacokinetics of oseltamivir at standard and high dosages, took up seven journal pages and 8,545 pages of the clinical study report.
But the researchers were also shaken, they said, by the “fragility” of some of their assumptions.
For instance, they found that the clinical study reports showed that in many trials, the placebo contained two chemicals not found in the oseltamivir capsules.
“We could find no explanation for why these ingredients were only in the placebo,” they wrote in BMJ, “and Roche did not answer our request for more information on the placebo content.”
Jefferson and colleagues also reported they found disparities in the numbers of influenza-infected people reported to be present in the treatment versus control groups of oseltamivir trials.
One possible explanation, they noted, is that oseltamivir affects antibody production – even though the manufacturer says it does not.
Gaps in Knowledge Remain
That question is profoundly important, Doshi told MedPage Today, because it may offer clues to how the drug works – one of the gaps in knowledge about oseltamivir.
“You can’t make good therapeutic decisions if you don’t know how the drugs works,” he said – information that he and his colleagues suspect may be buried in the mass of missing data.
It’s also important, he said, because public health agencies have been making decisions to stockpile oseltamivir without a clear understanding of the facts.
Essentially, he said, those decisions have been based on the flawed study – a Roche-supported meta-analysis – that was thrown out of the 2009 Cochrane review.
“They’re taking the drug manufacturer’s word at face value,” he said.
The results seem unlikely to resolve conflicts over the medical value of the drug, which is a major cash cow for Roche, adding some $3.4 billion to the company’s bottom line in 2009 alone, according to Deborah Cohen, investigations editor of BMJ.
In an accompanying article, Cohen said that “clinicians can be forgiven for being confused about what the evidence on oseltamivir says.”
She noted that the European Centre for Disease Prevention and Control, the CDC, and the World Health Organization “differ in their conclusions about what the drug does.”
As well, those conclusions are often contradicted by claims on the drug labels – themselves allowed by regulators, Cohen argued.
The Cochrane reviewers reported grant support from the U.K. National Institute for Health Research and Jefferson and Doshi reported they had no recent financial links with industry.
Cohen is employed by BMJ.
After reviewing the results of two 
Atencion Primaria 