Reproducibility Versus Replicability: Two Words Careful Readers Keep Apart

Reproducibility and replicability sound interchangeable, but a landmark national science report gave them separate jobs. Reproducibility asks whether the same data, run through the same analysis, produce the same numbers, which is mostly a test of transparency and record keeping. Replicability asks the harder question of whether an independent study, gathering new data, reaches a compatible conclusion. A finding can be flawlessly reproducible and still fail to replicate.

Two words that are not synonyms

In everyday conversation people use reproducible and replicable as if they mean the same thing. In the careful vocabulary of research integrity they do not, and the difference decides how much reassurance a claim actually gives you.

A consensus report from the National Academies set out to stabilize the terms precisely because their casual use was causing confusion across fields. The distinction it drew is simple to state and worth keeping in mind every time you read a validation claim.

What reproducibility actually checks

Reproducibility, in this framing, is about computation. Take the original dataset, apply the original code and analytical choices, and see whether the same figures and statistics fall out. If they do, the work is reproducible.

This is a real and valuable check, but notice what it tests. It tests whether the analysis was described honestly and completely enough that someone else can retrace it. It is a check on the paper trail, not on whether the effect exists in the world. Shared data and shared code are what make it possible.

What replicability actually checks

Replicability sets a higher bar. Here an independent team runs a fresh study, collecting new data under similar conditions, and asks whether the conclusion holds. If a comparable result appears, the finding replicates.

Because replication uses new data, it probes the claim itself rather than the bookkeeping. A direct replication copies the original methods closely; a conceptual replication tests the same idea a different way. Both ask the question that matters most to a reader deciding whether to believe a result.

Why a result can be reproducible but not replicable

These two properties can come apart, and that is the insight worth carrying away. A study can share its data and code, let anyone regenerate its exact numbers, and still describe an effect that vanishes when a new sample is collected.

That happens when the original result rode on chance, a small sample, or analytic flexibility. Reproducibility confirms the arithmetic was faithful to the data at hand. It says nothing about whether that data captured a stable truth. This is why perfectly transparent work still needs replication.

Why fields disagree about the words

The National Academies acknowledged that some disciplines use the two terms in the opposite order, which is part of why the report worked to fix definitions rather than assume them. Reading across fields, you cannot rely on the label alone.

A companion analysis by meta-research scholars separated the ideas further, distinguishing whether the methods, the results, or the inferences are what gets reproduced. The practical lesson is the same: the word is less informative than the specific thing that was repeated.

How to read the claim in practice

When a paper or a press note says a finding was reproduced, pause and ask a single question. Did someone rerun the original data, or did someone run a new study and see the same thing?

The first is reassuring about honesty and transparency. The second is reassuring about reality. Both are good, but they are not the same good, and a careful reader keeps them apart before deciding how much confidence a result has earned.