HHS says new vaccines should be tested against placebos. They already are

The U.S. Department of Health and Human Services’ potential plan to alter vaccine testing has baffled health experts, who worry it could reduce protection from infectious diseases and waste time and money.

The agency said in a recent statement that it intends to require all new vaccines be compared with inert placebos, like saline shots, in clinical trials. HHS claims the move will increase transparency and is a “radical departure from past practices,” according to a statement to Science News from an agency spokesperson.

But the safety and efficacy of genuinely new vaccines — that is, ones for diseases with no existing approved inoculations — are already assessed through placebo-controlled trials, says pediatric infectious diseases specialist Paul Offit. He directs the Vaccine Education Center at the Children’s Hospital of Philadelphia and is a member of the U.S. Food and Drug Administration’s vaccine advisory committee.

HHS is “offering this as some grand new venture, a new golden age of vaccinology,” he says. “We’ve been doing this since the ’40s.”

It also remains unclear which vaccines would count as “new” to HHS, which is headed by known vaccine skeptic Robert F. Kennedy Jr. If the agency considers new vaccines to include updates to keep pace with new or newly dominant strains of a virus, such as an annual COVID-19 shot, a clinical trial requirement would likely cause delays, Offit says. That would make updated vaccines “less available and more expensive without any critical additional information about efficacy or safety.”

Science News spoke with Offit and three other health experts to learn about existing processes that test and track the efficacy and safety of vaccines.

What do health experts consider to be new vaccines, and how are they tested?

For people who study and work with vaccines, a new one would be entirely novel — not a modified version of a previous vaccine, says biostatistician and clinical trials expert Susan Ellenberg of the University of Pennsylvania Perelman School of Medicine. “Those new vaccines are always tested against a placebo control,” says Ellenberg, who monitored the safety of COVID-19 vaccines developed through Operation Warp Speed, a program started under the first Trump administration.

That “new” label can also include a vaccine that uses a technology not found in any approved vaccines, Offit says. For instance, the flu vaccine Flublok, which was approved by FDA in 2013, does not involve the replication of flu viruses in chicken eggs to produce immunity-creating vaccine ingredients, a method regularly used since the 1940s. Instead, Flublok’s production uses insect cells to make certain flu virus proteins, which are included in the jab.

In addition to testing the vaccine candidate against saline as a placebo, Flublok went head-to-head with an already available flu shot. The clinical trials suggested the new vaccine had an improved ability to produce an immune response, better efficacy and similar side effects compared with the older version.

Are updated versions of vaccines, like annual flu and COVID-19 shots, evaluated the same way?

Because researchers have already figured out the safety and efficacy of those vaccines in previous studies — using placebos — more testing with placebos would be redundant, says bioethicist Seema Shah, director of research ethics at Lurie Children’s Hospital of Chicago.

Speed is crucial with rapidly evolving viruses, like those that cause COVID-19 and the seasonal flu, she says. HHS indicated to the Washington Post that the new requirement would not apply to flu shots, but focused concerns on COVID-19 vaccines.

To test vaccines altered only to include new virus strains, Shah says, researchers can examine proxies for immune protection to see whether the shot works as intended. For instance, the amount of infection-blocking antibodies a person’s immune system produces in response to a vaccine, measured from a blood draw, can indicate how much protection they’ve gained against that pathogen.

Introducing placebo-controlled trials at this stage would slow the availability of otherwise approved vaccines. And it would deny people in the placebo group an established method for protection, potentially leaving them to get sick more easily than those in the vaccine group, Shah says.

“It would be violating ethical standards and slowing down the process in a way that means we can’t compete with these infectious diseases,” she says.

Have the childhood vaccines recommended by the U.S. Centers for Disease Control and Prevention been tested against placebos?

Yes. These vaccines — including ones for polio, measles and chickenpox — have been tested against placebos in clinical trials, experts say.

Many trials used saline placebos to initially assess safety. Others involved placebos containing some components of the vaccine, minus the ingredients that create immunity. This can prevent participants and researchers from figuring out which group they’re in based on qualities saline wouldn’t replicate, such as known side effects, and it can reveal if the immunity-creating ingredients cause specific side effects.

Sometimes a different vaccine, also called an active control, is used instead of a placebo. That’s what happens when the comparator vaccine resembles the new vaccine in some way. For instance, new vaccines for human papillomavirus, or HPV, were compared with hepatitis vaccines because they shared similar formulations and presumable side effects, says public health scientist Jess Steier, founder and CEO of science communications company Unbiased Science, who is based in Pelham, Mass.

Moreover, if an established vaccine for the disease already exists, it would be unethical to withhold it when evaluating a different version, she says.

Vaccines often go through combinations of trials testing them against various placebos and controls. Researchers are compiling a list of these studies in response to HHS’s claim that, except for the COVID-19 vaccine, none of the CDC-recommended childhood immunizations were tested against inert placebos. That incorrect claim was included in the statement to Science News.

“We certainly have a long history of doing placebo-controlled trials,” Ellenberg says. The polio vaccine field trials of 1954 included more than 600,000 children randomly assigned to get either the new vaccine or a placebo. Another 1 million kids received the vaccine or no intervention, depending on which grade they were in. Clinical trials for COVID-19 vaccines have been some of the biggest placebo-controlled vaccine trials since then, she says, with participant numbers in the tens of thousands for some studies. 

A few clinical trials conducted decades ago investigating other vaccines involved smaller numbers of participants than are preferred nowadays, Ellenberg says. But “there’s still been enormous observational experience afterwards that if there had been something untoward, it would’ve popped up.”

How do researchers assess the safety of a vaccine before it’s available to the public?

Gaining FDA approval requires several steps for a brand-new vaccine. Before reaching humans, vaccine candidates undergo extensive research in animals, Steier says.

Then, there are multiple phases of clinical trials in people. “We move through very small groups of volunteers to look for those glaring safety signals, and then we gradually expand to larger groups,” she says. After establishing dosing, efficacy and safety — typically in adults — through three multiyear phases, FDA will decide whether to approve the new vaccine. If approved, the immunization gets tested in more vulnerable populations, like kids. “We have so many eyes on vaccines at all times,” Steier says.

FDA sets a high bar for vaccines, says Ellenberg, who assessed clinical trials at the agency’s Center for Biologics Evaluation and Research from 1993 to 2004. While a drug might gain approval for reducing the risk of death by 10 percent, she says, vaccines generally need to show they lower the risk of developing disease by at least 50 percent.

“Vaccines are given to healthy people, and they’re given to lots and lots of people,” Ellenberg says. So “it’s not worth [approving] unless it’s going to have a substantial effect.”

Is a vaccine’s safety tracked and investigated after it’s on the market?

Yes, through multiple routes. Vaccine manufacturers often conduct follow-up observational studies, in which they spend time monitoring vaccine recipients for any troublesome events, Ellenberg says. That’s what happened for 10 years with the first chickenpox vaccine, which was approved by FDA in 1995. Universities and health organizations outside of the pharmaceutical industry have surveillance systems as well, Steier adds.

Another method is through the Vaccine Adverse Event Reporting System, or VAERS, which is managed by FDA and CDC. Anyone can report what they suspect to be a vaccine-related event to the system, although many of the problems are known side effects, like fever, or have nothing to do with vaccination and happened to occur soon after receiving a shot, Ellenberg says. Unfortunately, that can include death and disability in infants, because the first year of life — when babies receive many immunizations — is one of the most vulnerable periods and when preexisting genetic conditions can start showing up.

“When you vaccinate 4 million babies a year, of course some of those things can happen by chance,” Ellenberg says.

VAERS’ large swath of information has flagged real problems. The first rotavirus vaccine, which became available to U.S. infants in 1998, caused kinks in the intestines and abdominal pain, Ellenberg says. Reports to VAERS raised the issue, which researchers investigated, and the vaccine was taken off the market the following year.

CDC also sponsors the Vaccine Safety Datalink, or VSD, which tracks vaccination information collected by participating health care systems. That health data can reveal whether a rare side effect is happening more often in people who received a certain vaccine compared with people who did not, Offit says. The VSD helped uncover that mRNA COVID-19 vaccines cause inflammation of the heart in about one in 50,000 people, which would be difficult to detect in clinical trials because of the small number of participants relative to all vaccine recipients, he notes.

The HHS spokesperson says the agency is building surveillance systems that will accurately measure the risks and benefits of vaccines. But the several systems currently in place are already working as they should, Offit says. “People are constantly looking for safety problems.”