Image: Alexandra_Koch / Pixabay
People around the world believe the several promising COVID-19 vaccines were designed to do the two things we most want them to do: reduce severe illness and death, and reduce the spread of the disease.
But a central problem, experts say, is that none of the new vaccine trials were, in fact, designed to achieve either of those goals — but rather merely to show that they could reduce symptomatic cases of COVID-19, no matter their severity.
And investigative health reporter Serena Tinari says a related problem is that many reporters don’t know what the trials by, for instance, Pfizer and BioNTech, Moderna, and AstraZeneca were seeking to achieve, and that media coverage of their claimed Phase 3 interim results has generally failed to question the efficacy of the vaccines on those two critical outcomes.
The reasons for this, she says, include the fact that the pandemic has forced journalists around the globe to become health reporters almost overnight; that the “fast track” approval model has propelled pharmaceutical development many times faster than normal; and, above all, that none of the trial data has been made public.
Tinari, co-founder of nonprofit health watchdog Re-Check.ch, spoke in a GIJN webinar entitled “Behind the COVID-19 Vaccine Race.” Dr. Peter Doshi, PhD, an associate professor of pharmaceutical health services research at the University of Maryland School of Pharmacy, and an expert in the drug approval process, joined Tinari; 169 journalists from 59 countries attended the webinar.
Doshi is also an associate editor at the peer-reviewed medical journal BMJ, where he recently set out his concerns about the trial protocols in an article entitled Will COVID-19 Vaccines Save Lives? Current Trials Aren’t Designed to Tell Us.
“There are reasons to be concerned — in this unprecedented time, I think we should be demanding nothing less than public access to all the data,” Doshi said, echoing a call he made for transparency in a BMJ blog post. “These trials have been designed to reduce symptomatic COVID of any severity; what they are not designed to provide is definitive evidence regarding whether they reduce the risk of severe COVID, hospitalization, ICU use, or death.”
Doshi says the companies did nothing wrong in pursuing the same, modest goal. Rather, he says the US regulator, the Food & Drug Administration, offered manufacturers two standards to choose from in June 2020 — either a 50% reduction in lab-confirmed symptomatic COVID-19, or a 50% reduction in coronavirus infections — “and the companies all chose the first option.”
But wouldn’t protection against mild disease protect at the same rate against severe forms of that same disease?
Not necessarily, says Doshi.
“You might think, ‘What does this matter? — if you can stop mild disease, then surely that will prevent severe disease as well,’ because first you get infected and then you get symptoms and then it could get worse,” he said. “But while that’s true for a single individual, that’s not the case when you have many people, as in a trial. You can have a situation where you have an overall reduction in mild cases, but no reductions in hospitalizations, ICU use or deaths.”
To illustrate how preventing mild disease does not always prevent severe forms of the same disease, Dr. Peter Doshi offers this hypothetical vaccine trial — showing 60% effectiveness in overall case reduction, but no difference in the number of severe cases or deaths. Image: Courtesy Dr. Peter Doshi
Tinari and Doshi agree that one piece of good news about the vaccine pre-approval process is that most companies have already published their study protocols — unusual at the Phase 3 stage — and that reporters can use these to ask questions about crucial outcomes.
For Doshi, key facts still unknown about the various candidate vaccines include:
- Whether – and by how much – they reduce the risk of hospitalization or ICU admission;
- Whether – and by how much – they reduce the risk of transmission;
- How well they protect the most vulnerable groups, like the elderly;
- How well they work in children and adolescents;
- How well they work in immunocompromised people;
- How effective they will be after three, or six, or 12 months — given that, with other vaccines like those for influenza, the immune response can drop over time.
Doshi warns that some facts about clinical efficacy — such as a vaccine’s ability to save lives — might never be known, if vaccines are approved prematurely. For instance, he says that randomized controlled trials — which provide the most accurate data on efficacy — would likely not be conducted after vaccine approval for ethical reasons.
“This may very well be the last chance we have to see how the product works compared to placebo,” he said.
But Doshi says journalists need to be aware that it is not too late to find answers to the biggest questions about the vaccines: “There is a way to get vaccines to people without compromising the trials, and given the demand and the urgency, it’s important to know there are mechanisms to do so. Access does not require approval. In the US, it’s called the extended access program, also called compassionate use. This could start today — we can and should be talking about this much more.”
Asked about the steep learning curve for non-health reporters covering the vaccine race, Tinari says detailed tips on health reporting, as well as some 500 links, are available on the just-released GIJN guide, Investigating Health and Medicine, which Tinari co-authored with investigative journalist Catherine Riva.
Tinari says there will be many areas for investigative reporters to focus on during the coming rollouts — including conflicts of interest, the credibility of government advisors, and rollout accountability — but that “the biggest issue right now is open access to all the data.”
“We need people like Peter Doshi to be able to analyze all the evidence — we can’t just rely on a press release,” she said. “And unfortunately, though regulators basically do what they can, this is also a very toxic environment now, because the regulators and governments are very much under pressure, and that’s very dangerous.”
Additional Reading
Document of the Day: A Look at Pfizer’s Vaccine Press Release
Top 10 Tips for Investigating Health Care, from the Experts
Rowan Philp is a reporter for GIJN. Rowan was formerly chief reporter for South Africa’s Sunday Times. As a foreign correspondent, he has reported on news, politics, corruption, and conflict from more than two dozen countries around the world.
As Dr. Doshi said, it is important to evaluate the results of a trial in light of what the trial was designed to investigate, ie, the outcomes measures. These measures are listed in the trial protocol, an abbreviated version of which is available at clinicaltrials.gov for the vast majority of trials in all medical fields. Also listed are the inclusion and exclusion criteria for trial participation, which are important for evaluating the applicability and generalizability of the trial results to populations either represented or not reperesented in a trial. In addition, Dr. Doshi’s example (see image) points up the necessity of looking at absolute numbers rather than relative numbers of outcomes. If a vaccine is 60% effective in preventing cases, but if susceptible people (elderly, co-morbidities, etc) are more likely to be in the remaining 40% who get infected, they may be more at risk for severe disease. So it is possible that the vaccine may have no effect on hospitalizations or deaths, as in the illustration, even though it worked great in preventing disease for the generally healthier people in the population.