A large Danish study recently provided reassurance that aluminum-containing vaccines are not associated with increased rates of chronic health conditions in children, including autism. But Health and Human Services Secretary Robert F. Kennedy Jr. misrepresented the study’s findings, claiming that the paper’s supplementary data “shows calamitous evidence of harm.”
The study, published on July 15 in Annals of Internal Medicine, drew attention because of its large size and rigorous methods. Anti-vaccine activists, however, quickly seized on it to claim that it was flawed, particularly after the journal mistakenly uploaded an earlier version of the supplementary data.
Kennedy declared in an X post last month that the authors used a “long parade of statistical artifices” to “achieve their deceptive results,” incorrectly alleging that the study actually showed an association between vaccines and autism. He linked to an article with his byline on TrialSiteNews, which consisted of a list of criticisms of the study peppered with unfounded accusations on the researchers’ motives. It called for the journal to “immediately retract this badly flawed study.”
The journal, however, defended the study. Dr. Christine Laine, the editor-in-chief of Annals, wrote in an editor’s response that the journal found the study “to be among the strongest research currently available” on the subject and that there was no basis for retraction.
Small amounts of aluminum are used in some vaccines as adjuvants that boost the immune response and make vaccines more effective. Among those that contain aluminum are vaccines that protect against hepatitis A and B, HPV, and diphtheria and tetanus. There is no aluminum in the measles, mumps and rubella vaccine, which has been subject to unfounded claims about autism in the past.
Aluminum has been used as a vaccine adjuvant since the 1920s. The quantities of aluminum in vaccines represent an “extremely low risk to infants,” researchers from the U.S. Food and Drug Administration have concluded. Despite this, aluminum has long been a target of anti-vaccine groups.
In the case of the Danish aluminum study, Kennedy dismissed the results of the paper’s main analysis. It found no relationship between aluminum exposure from vaccines and an elevated risk of dozens of conditions, including autism. Instead, he zeroed in on results on autism in two of the paper’s 15 supplementary figures and tables, which he claimed on X were “a devastating indictment of aluminum-containing vaccines directly contradicting the published study’s conclusions.”
As we will explain, Kennedy’s interpretation of these figures is unwarranted. For one, statisticians know that if a paper makes a large number of comparisons, some results are expected to be statistically significant by chance alone. It is inappropriate to focus on a single result or subset of results while ignoring their context.
Kennedy “is doing extreme cherry-picking of the results he likes and ignoring and dismissing all of the results he doesn’t like,” Jeffrey S. Morris, director of the division of biostatistics at the University of Pennsylvania’s Perelman School of Medicine, told us.
For another, Kennedy highlighted an analysis that the authors of the paper and other experts said should not be used to determine whether aluminum in vaccines causes health conditions.
“In short, Kennedy is cherry-picking a fragile secondary analysis that is explicitly disclaimed by the study’s authors and misrepresenting it as a refutation of the main findings,” vaccinologist Helen Petousis-Harris of the University of Auckland told us in an email. “It’s a classic misuse of supplementary data.” She called the supplementary results Kennedy highlighted a “statistical blip, not a bombshell.”
Anders Hviid, head of the epidemiology research department at the Statens Serum Institut in Denmark, agreed that Kennedy’s interpretation of his team’s study amounted to cherry-picking. “Our results showed that increasing exposure to aluminum from vaccines was not associated with increasing risks of a wide range of early childhood health conditions in Denmark,” he told us in an email.
Kennedy’s statements followed a pattern familiar to those who have tracked the secretary’s history of unfounded claims on vaccines and autism. Historically, Kennedy has ignored or attempted to discredit well-done scientific studies that have failed to show a link between autism and vaccination. His criticisms are often accompanied by inflammatory language and unfounded claims about researchers’ motivations. At the same time, he has zeroed in on results that he claims show an association, elevating flawed studies and highlighting decades-old unpublished statistics that weren’t borne out by later analysis.
HHS did not reply to a request for comment.
‘The Cherry-Pick of All Cherry-Picks‘
Denmark’s universal, publicly funded health care system and detailed record-keeping allow researchers to study the relationship between medical care and health outcomes. Hviid and his colleagues analyzed vaccination and health data on more than 1.2 million children who were born in the country over the course of around two decades. As the Danish vaccine schedule changed over time, the amount of aluminum in recommended vaccines also changed, allowing the researchers to assess whether each additional milligram of aluminum exposure had any relationship to chronic disease.
In their primary analysis, the researchers found no link between the amount of aluminum the children received via vaccination and elevated rates of 50 chronic conditions. These conditions included asthma and allergic diseases, autoimmune diseases, and neurodevelopmental conditions. This last group of conditions included attention-deficit/hyperactivity disorder and five autism-related diagnoses.
“The main findings, based on solid methods and all the data, showed no increased risk,” Petousis-Harris said (emphasis is hers).
Rather than highlighting these reassuring results, Kennedy focused on a supplemental figure showing a barely statistically significant increase in one autism-related diagnosis in a single set of children.
“The data show a statistically significant 67% increased risk of Asperger’s syndrome per 1 mg increase in aluminum exposure among children born between 2007 and 2018,” Kennedy wrote on X. (In the U.S., Asperger’s syndrome is no longer a specific diagnosis. The condition was folded into the autism spectrum disorder diagnosis in 2013 in the fifth edition of the American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders.)
“That is the cherry-pick of all cherry-picks,” Morris said.
The figure was part of a series of analyses meant to look at the data in various ways and see if the primary results held up — and hold up they did, in almost all cases. There was no increased risk of any other autism-related diagnosis in the analysis when dividing the children by birth year, nor of Asperger’s in children born in the first 10 years of the study, although Kennedy failed to mention these results. Nor was there an increased risk of autism, including Asperger’s, in the primary analysis nor in a variety of other analyses, such as when the researchers divided the children by sex or changed the length of time they followed the children’s health.
“You can’t just go through pages and pages and pages of tables and pick one little one out that you like and ignore all the others,” Morris said.
Moreover, the Asperger’s statistic Kennedy highlighted was imprecise due to the small number of children diagnosed with Asperger’s in the relevant time period. Statistics often come with confidence intervals, which are calculations of the range of answers one would expect to come up with if repeating a study many times. The smaller the group of people researchers are looking at, the wider the confidence interval tends to be and the less precise the result.
In this case, there were only 51 children diagnosed with Asperger’s with birth years in the 10-year period the researchers were looking at. The estimate of a 67% increased risk of Asperger’s came with a confidence interval indicating a 95% chance that the actual answer was somewhere between a 1% increased risk and a 177% increased risk. That is to say, there was a great deal of imprecision in the result.
Furthermore, Morris explained, the likelihood of getting a statistically significant result by random chance alone increases the more comparisons a person makes. Typically, researchers consider a result statistically significant if there’s just a 5% probability that the result occurred due to chance alone. But Morris said that this means that even in a situation in which there is no difference between the groups being compared, one can expect one out of every 20 comparisons to give a false positive. Thus, it is not surprising that someone looking for a particular result in a paper with many comparisons will find validation for their view, even if the overall data do not support it.
“Several comments noted marginally significant associations with some neurodevelopmental conditions observed in secondary analyses reported in the supplement,” journal editor-in-chief Laine wrote in her editor’s response. “Secondary analyses explored over 540 comparisons, so it is expected that some are likely to be statistically significant by chance alone. The editors agree with the authors’ approach of basing conclusions on the primary adjusted analyses that are the most rigorous and valid.”
Unwarranted Interpretation of Data
Kennedy also highlighted a second set of results in the paper’s supplementary materials to justify his claim that the paper showed an increased risk of autism. But Hviid told us that not only was this set of results cherry-picked, but it shouldn’t have been used to draw conclusions on causes of disease in the first place.
The figure — which divides kids’ aluminum exposure into highest, medium and lowest categories — was “not introduced to test the hypothesis of the paper, and it certainly does not ‘overrule’ the main paper and its approach,” Hviid said.
Kennedy, however, portrayed the results as showing an increased risk of autism. “Compared to the moderate exposure group, for every 10,000 children in the highest aluminum exposure cohort, there were: 9.7 more cases of neurodevelopmental disorder, 4.5 more cases of autistic disorder, and 8.7 more cases of the broader category of autism spectrum disorder,” Kennedy said on X.
Petousis-Harris said that this supplementary figure does not indicate an increased risk of neurodevelopmental disorders due to aluminum exposure, contrary to Kennedy’s claim. “The authors and journal editor both said these supplementary results are invalid for causal interpretation and shouldn’t be used the way Kennedy is using them,” she said. “This is like trying to win a soccer game by picking up the ball and running it into the goal—after the referee said ‘no hands allowed.’”
Hviid and his colleagues said in an Aug. 11 follow-up comment on the paper that their primary analysis “is the most appropriate for evaluating the hypothesis, and we stand by our interpretation.”
In their primary analysis, the authors adjusted for birth year because birth year can independently influence both vaccine aluminum exposure and a child’s likelihood of getting a certain diagnosis. For instance, in the case of autism, we know that diagnoses have increased over the years at least in part due to changes in screening, broadening diagnostic criteria and increased awareness.
In the figure Kennedy highlighted, it was impossible for people born prior to 2002 to fall into the highest category of aluminum exposure, the researchers explained. This violates a statistical principle called the positivity assumption, they said. It’s not valid in this type of analysis to compare groups of people if there is an important variable for which there are no people represented in one of the groups.
“If you’re going to adjust for a confounding factor that affects the three groups, in this case, you need to have that factor at least represented somewhat in each of the three groups or you can’t correct for it,” Morris explained.
“[Y]ou can’t compare apples to oranges if you don’t have any oranges in the same basket,” Petousis-Harris said, adding that when the positivity assumption is violated, “estimated associations can become highly unstable and misleading.”
When the researchers did the same analysis only looking at children born in 2002 and beyond — a group in which all three aluminum exposure levels were represented — they did not find significant differences in autism diagnoses between groups.
Mixing Criticism With Unfounded Accusations
In a familiar pattern, Kennedy mixed reasonable and unreasonable critiques of the study with unfounded and inflammatory claims about the study authors’ motives.
All scientific studies have some limitations. The Danish study was observational, meaning that it analyzed data on health without manipulating the exposures, as would be done in a randomized, controlled trial. It would be unethical to randomly assign people to receive or not receive childhood vaccines known to be effective. But in observational studies, there is always the possibility that people in different groups share some trait that influences the outcome.
To give one example, Kennedy brought up the idea that, by adjusting for general practitioner visits before age 2, the researchers obscured signals of harm in children whose doctor visits were caused by vaccine injuries. Researchers adjust for health care visits because patients who visit their doctor more are also more likely to successfully get a diagnosis for their health problems. Responding to that criticism, Hviid and his colleagues shared an analysis in an Aug. 11 comment showing that there remained no significant link between aluminum and harms even after taking away this adjustment.
Many of Kennedy’s points are “reasonable to raise and discuss” Hviid wrote in a response published Aug. 3 on the same website. But Kennedy’s criticisms are “couched in needlessly vitriolic rhetoric,” Hviid wrote.
For one thing, Kennedy mischaracterized the history of the study’s supplementary materials. “Fierce criticism from the scientific community has now forced the authors to release their supplementary data, which shows calamitous evidence of harm,” he wrote on X.
We have not been “‘forced’ to release anything,” Hviid told us in an email. The paper was published with supplementary materials from the beginning, but the journal “put up an older version of the supplementary materials as a simple mistake,” Hviid said, and the journal corrected this two days later.
Laine, the journal’s editor in chief, explained that Hviid and his co-authors had found during the review process that the the health data they had used was missing some data from psychiatry hospitals, so they redid their analysis using the additional data. But journal staff accidentally uploaded out-of-date supplementary materials, rather than the updated materials the authors had submitted. “Annals published the correct supplement as soon as the mistake was detected,” Laine said. “This was a simple administrative error and reflects no malfeasance on the part of the authors or Annals.”
Kennedy also repeatedly implied some ill intent or financial motive on the part of Hviid and his co-authors, despite a lack of evidence for these accusations. He opened his article, for instance, by calling the study “so deeply flawed it functions not as science but as a deceitful propaganda stunt by the pharmaceutical industry.” He went on to say, again without evidence, that the “architects of this study meticulously designed it not to find harm.”
“The claim that we have designed this study to find no association can be easily refuted,” Hviid wrote in his response. He explained that the design was based on a 2022 U.S. study — funded by the Centers for Disease Control and Prevention, with some agency co-authors — that showed a possible link between aluminum-containing vaccines and asthma. Kennedy had mentioned this study in his TrialSiteNews article as the single citation supporting his claim that there are “mountains of contrary literature documenting the neurotoxicity of aluminum and its association with autoimmune and allergic diseases.”
The U.S. researchers asked Hviid to try to replicate the finding on asthma and aluminum using the Danish data, and his team did not find a link between aluminum-containing vaccines and asthma. He presented these results at a 2023 meeting of the CDC’s vaccine advisory committee. For the new paper, Hviid and his colleagues used the same basic approach to assess a wider range of chronic conditions.
As justification for his unfounded claim that the study results were driven by pharmaceutical companies, Kennedy wrote that three study authors, including Hviid, “are affiliated with Denmark’s Statens Serum Institut (SSI), a government-owned vaccine company that develops a number of aluminum-containing vaccines.”
In fact, as Hviid explained in his Aug. 3 response to Kennedy, Statens Serum Institute — which translates to State Serum Institute — is not a company, but rather is Denmark’s “national communicable diseases control institute,” and does not currently produce or sell any vaccines.
Previously, the institute manufactured aluminum-containing and other vaccines, including those protecting against diphtheria, tetanus and whooping cough, as well as polio and tuberculosis. But the Danish government eventually decided that manufacturing vaccines was “no longer necessary nor profitable at a national level” and sold its vaccine production capabilities in 2017.
Kennedy also wrote in his article that Hviid’s funding from the Novo Nordisk Foundation “call into question the study’s independence.” The Novo Nordisk Foundation is a nonprofit organization that controls the Danish pharmaceutical company Novo Nordisk, known for its diabetes and anti-obesity drugs.
Hviid pointed out in his response to Kennedy that his grants from the foundation are unrelated to the vaccine research and were instead related to heart failure and machine learning. “The Novo Nordisk Foundation is one of the biggest supporters of health science in Denmark,” he added.
In addition, Kennedy claimed that because the researchers did not share the raw Danish health care data, the study was lacking in “transparency and reproducibility.” But as Hviid explained, sharing the raw data is impossible due to regulations protecting sensitive health data of Danish individuals.
Petousis-Harris disputed the notion that the publication was lacking in transparency. “I would go so far as to say the authors have been exceedingly transparent in responding to critics, which stands in stark contrast to the mischaracterisation of the data by Kennedy. Like any study, this one is not without limitations,” she said, pointing to her own blog post that discussed some of them. “However, it is an important contribution to the body of evidence.”
Editor’s note: FactCheck.org does not accept advertising. We rely on grants and individual donations from people like you. Please consider a donation. Credit card donations may be made through our “Donate” page. If you prefer to give by check, send to: FactCheck.org, Annenberg Public Policy Center, P.O. Box 58100, Philadelphia, PA 19102.