What the Big Replication Projects Actually Found

When research teams systematically repeated published studies, a large share did not replicate. In one psychology project, nearly all original studies had been statistically significant, yet only about a third of the replications were, and effect sizes shrank by roughly half. A cancer biology project found replication effects far smaller than the originals. These numbers describe the state of a literature, not the guilt of any single paper.

Why anyone repeated a hundred studies

For years the incentive in science ran toward novelty, and repeating someone else's experiment was quietly undervalued. A handful of large, coordinated efforts set out to change that by doing the unglamorous work of rerunning published findings at scale.

The goal was not to expose individuals. It was to estimate a property of the literature as a whole: if you pick published, significant results and test them again with care, how often does the effect come back?

The psychology numbers, read carefully

The most cited effort repeated one hundred studies from three psychology journals, using high-powered designs and, where possible, the original materials. About ninety-seven percent of the original studies had reported statistically significant results.

Among the replications, roughly thirty-six percent reached significance, and the replication effects were on average about half the magnitude of the originals. Tellingly, the best predictor of whether something replicated was the strength of the original evidence, not the personalities or skill of either team.

The cancer biology numbers

A parallel project turned the same lens on preclinical cancer research, repeating selected experiments from high-impact papers. It examined fifty experiments drawn from twenty-three papers, generating data on well over a hundred distinct effects.

For the positive effects that could be compared numerically, the median effect size in the replications was around eighty-five percent smaller than in the originals, and the large majority of replication effects were smaller than the effect they were testing. Combining the various methods it used, fewer than half of the effects replicated on the strictest counts.

Replication failure is not the same as fraud

It is tempting to read these figures as an indictment of the scientists involved. That reading is usually wrong. Effects shrink on repetition for reasons that have nothing to do with dishonesty.

Small original samples exaggerate whatever they happen to catch. Studies that reached significance are more likely to be published in the first place, which inflates the pool. And subtle differences in materials, populations, or conditions can genuinely change an effect. A meta-research essay had predicted much of this pattern in advance, from the mathematics of power and bias alone.

What a failed replication does and does not prove

The cancer biology team was careful to state the logic in both directions. A successful replication does not definitively confirm the original theory, and a failure to replicate does not disprove the original finding.

What a failure does is raise the burden of proof. It signals that more investigation is needed before a result can be treated as reliable. Reading these projects honestly means resisting both the urge to declare a field broken and the urge to wave the numbers away.

How to use this when you read one study

The practical takeaway is not despair; it is calibration. A dramatic new result, especially one with a small sample and a surprising effect, is a hypothesis worth watching rather than a settled fact.

Before you act on a single striking paper, look for independent replication, larger samples, and preregistered designs that limit analytic flexibility. The replication projects did not tell us that science fails. They told us how much weight one study can safely bear, which is less than a headline usually implies.