What is actually the emerging evidence about non-specific vaccine effects in randomized trials from the Bandim Health Project?

Background and Purpose

The concept of NSEs suggests that some vaccines, especially BCG and measles-containing vaccines, may have beneficial or harmful effects on health outcomes beyond their specific target diseases. The Bandim Health Project in Guinea-Bissau has led research on NSEs; however, recent news coverage and scientific scrutiny in Denmark have raised questions about research practices within the project. The present commentary aims to re-evaluate the evidence from Bandim’s RCTs independently.

Methods

• A systematic review was conducted of 40 RCT publications from the Bandim Health Project; 26 were included after confirming trial registration and availability of prespecified outcomes.
• All p-values from tables and figures, or estimated from reported confidence intervals, were extracted.
• The authors applied the Holm-Bonferroni (HB) procedure for family-wise error rate (FWER) control, and the Benjamini-Hochberg (BH) procedure to manage false discovery rate (FDR), correcting for multiple testing both within individual tables/figures and across the entire set of studies.
• Conclusions in published papers were evaluated against six predefined criteria, checking whether findings deemed significant after correction still supported claims of NSEs.
• A meta-analysis using z-curve methodology assessed the strength and replicability of significant findings across all studies.

Main Results

• Only 12 of 26 included RCTs explicitly reported on the primary prespecified outcome; of these, just one showed a statistically significant primary result.
• Across 25 studies with null primary outcomes, there were 1481 hypothesis tests; 215 p-values were below 0.05 before correction.
• When controlling for multiple comparisons, just one secondary outcome in one paper remained statistically significant; the evidence for NSEs vanished in all other cases after correction.
• Authors often promoted secondary findings as causal even when primary endpoints showed no effect; this raises concerns about selective reporting and confirmation bias.
• Analysis of z-scores and replication rates revealed that only about 1 in 4 significant findings would be expected to replicate, suggesting low mean study power and potential reporting biases.
• The observed discovery rate exceeded the expected power, hinting at overreporting of positive results relative to expected statistical validity.
• The commentary concludes that, based strictly on RCTs conducted under the Bandim umbrella, there is no strong or consistent confirmatory evidence supporting robust NSEs for BCG or measles vaccines.

Opinion

It sounds like a miracle, yet ‘believers’ in NSE trouble vaccinologists around the globe for decades, that such effects exist. The issue is that these proposed effects mostly lack biological plausibility and instead rely on statistical correlations from observational studies. These in principle exclude the option to evaluate cause and effect: as the numbers of storks decline, a significant correlation with the number of newborns does not allow the conclusion that storks deliver newborns. The article above underscores the importance of methodological rigor and transparent reporting in vaccine research, especially when substantial policy recommendations are at stake. It serves as a cautionary tale: even well-cited research programs can overstate findings if secondary or exploratory results are substituted for primary endpoints without proper correction.

By Health Literacy Asia