The Fragility Index: How Many Events Separate a Positive Trial From a Null One

The fragility index is the smallest number of patients who would have to switch from a non-event to an event (or the reverse) for a statistically significant trial result to stop being significant. You take the trial's 2x2 table of outcomes, move one patient in the group with fewer events from "no event" to "event," recount the p-value, and repeat until significance disappears. The number of switches it took is the fragility index. A large index means the result rests on many events and is hard to overturn; a small one means a handful of patients, sometimes fewer than the number lost to follow-up, carried the whole conclusion. This piece is general education and not medical advice; for decisions about your own care, talk with a clinician who knows your history.

I read trials both as a reviewer and as someone who has produced them, including a meta-analysis I co-authored in Diabetes Care and randomized work comparing insulin formulations. The fragility index is one of the quieter tools in evidence appraisal, and one I wish more readers reached for before they call a result "proven."

What the fragility index actually measures

Picture a trial that reports a p-value just under the conventional 0.05 threshold. That single number tells you the result cleared a bar, but not by how much, and not how many people it took to get there. The fragility index reframes the same data as a physical question: how many individual patients would need a different outcome before the finding evaporates?

The arithmetic is deliberately concrete. You start with the group that had fewer events, convert one non-event into an event, and rerun the significance test. If the result is still significant, you convert another. The count at which significance breaks is the index. Because you are moving real people between cells of the table, the answer comes in units anyone can hold: patients, not probabilities.

A companion figure, the fragility quotient, divides the index by the sample size. The same handful of switches means one thing in a small trial and something different in a large one, and the quotient keeps that context attached.

Why a significant trial can hinge on a few events

Significance testing rewards trials for crossing a line, and it says nothing about the margin. A study can land just under the threshold or far below it, and in a headline both read as "positive." Those two results are not equally trustworthy, and the fragility index is one way to see the gap.

Small event counts are where fragility lives. Many trials are powered to detect a difference in a relatively rare outcome, so the number of actual events, the thing significance depends on, is far smaller than the number of people enrolled. When events are scarce, moving a small number of them can swing the test. That is not a flaw in any single study so much as a feature of how thin the evidence gets near the threshold.

Here is the comparison that tends to stay with people. A common trap is a trial in which more patients were lost to follow-up than the fragility index itself. If those who dropped out had their outcomes never recorded, and the result would have flipped after only a few outcome switches, the missing data alone could have carried the conclusion either way. The significant p-value looks solid until you set it beside the count of patients whose fate you never learned.

What the fragility index does not tell you

It is easy to treat a low index as a verdict of "wrong," and that is a mistake. Fragility is not the same as falsehood. A true effect measured in a small trial can have a small index simply because the trial was small; the effect may be entirely real and later confirmed. The index measures how much a result depends on a few data points, not whether the result reflects reality.

The tool has real limits. It is defined for two-group trials with a binary outcome and a significance test, so it does not apply cleanly to time-to-event analyses, continuous outcomes, or more complex models without adaptation. It also inherits the arbitrariness of the 0.05 threshold it is built around; move the line and you move the index. And it is no substitute for thinking about bias or confounding, which can distort a result no matter how many events support it. A robust index on a biased trial is still a biased trial.

There is also a subtle asymmetry. The index usually counts switches in one direction, from non-event to event in the smaller-event group, which is the direction that erodes significance. That convention is reasonable, but it means the number describes fragility toward the null, not the full landscape of how the result could move.

How it complements the p-value and confidence interval

These tools answer different questions, and the fragility index earns its place precisely because it does not replace the other two. A p-value asks whether the data are surprising under a hypothesis of no effect. A confidence interval asks how large the effect plausibly is and how much uncertainty surrounds it. Neither speaks in units of patients, and neither foregrounds how thin the margin is.

The fragility index translates the same evidence into a count you can weigh against the messiness of the trial: the dropouts, the protocol deviations, the handful of adjudicated events that could have been coded differently. When the index is smaller than the number of patients lost to follow-up, that is a signal to read the methods with more care, not to discard the trial.

The honest reading is the combined one. A narrow confidence interval that excludes no effect, a small p-value, and a fragility index comfortably larger than the trial's missing data together tell a coherent story of a durable result. When those three disagree, the disagreement is the finding. A result that is significant but fragile is an invitation to look for confirmation, not a conclusion to bank on. Used that way, the index does what good appraisal tools do: it slows you down at exactly the moment a single number was tempting you to speed up.