Difference-in-Differences and Interrupted Time Series: Reading Quasi-Experiments

When a policy or program is switched on at a known moment and randomizing is impossible, researchers reach for quasi-experiments. Interrupted time series tracks one population's outcome across many time points and looks for a jump or a change in slope at the moment the policy started. Difference-in-differences compares the before-and-after change in a group that got the policy with the change in a similar group that did not, subtracting out whatever was happening to everyone. Each design leans on one untestable assumption about what would have happened otherwise, and that assumption is where a careful reader looks first.

When you cannot randomize a policy

A great deal of what shapes health is decided at the level of a law, a coverage rule, or a program that is rolled out to everyone at once. You cannot randomly assign a state to pass a tax, or a hospital to adopt a rule. Quasi-experimental designs exist to estimate the effect of such changes using the natural timing of when they happened and who they touched.

Two designs dominate this space. Interrupted time series watches a single population over time. Difference-in-differences compares a group that was affected with one that was not. They can be used separately or combined, and both are attempts to reconstruct the counterfactual, meaning what the outcome would have been if the policy had never arrived.

Interrupted time series

Interrupted time series takes repeated measurements of an outcome at regular intervals, many points before the intervention and many after, and fits a model that allows the line to change at the moment the intervention took effect. The usual tool is segmented regression, which can capture two kinds of change: a level change, an immediate jump up or down right after the intervention, and a slope change, a shift in the trend that unfolds over time.

The design is only as good as the comparison it makes against the projected old trend. Several technical threats can distort it. Points close in time are correlated, called autocorrelation, which can fool the model into seeing precision that is not there. Seasonal patterns can mimic or mask an effect. And any other event that happened around the same time, a co-intervention, competes for credit. Good analyses propose the expected shape of the effect in advance and adjust for season and autocorrelation rather than discovering the pattern after the fact.

Difference-in-differences

Difference-in-differences needs two groups and two periods. One group is exposed to the policy and the other is not, and you observe both before and after. You compute the change over time in the treated group, compute the same change in the comparison group, and take the difference between those two changes. That double subtraction is the whole idea.

Its power is in what the subtraction removes. Any fixed difference between the groups that does not change over time cancels out, and any trend that both groups share, such as a nationwide secular improvement, is subtracted away by the comparison group. What remains is meant to be the part of the change unique to the treated group at the time the policy hit, which is attributed to the policy.

The one assumption each design rests on

For interrupted time series, the crucial assumption is that the trend already in motion before the intervention would have continued unchanged if the intervention had never happened. The projected old line is the stand-in for the counterfactual, so anything else that bent the trend at the same time undermines the estimate.

For difference-in-differences, the crucial assumption is parallel trends: absent the policy, the treated and comparison groups would have moved in parallel, their gap staying constant. If the comparison group was already drifting toward or away from the treated group for its own reasons, the method will read that drift as an effect. Neither assumption can be verified directly, because both are statements about a world that did not happen.

How to pressure-test them

Because the key assumptions are untestable head-on, credible studies test them sideways. For difference-in-differences, the standard move is to plot the two groups in the pre-period and check that they really were tracking in parallel before the policy; a visible divergence beforehand is a red flag. Researchers also run placebo tests, pretending the policy started at a fake earlier date to confirm no effect appears where none should.

For interrupted time series, strength comes from a control series, a comparable population that did not get the intervention, or a control outcome that the intervention should not affect. If the effect shows up only in the treated series and only for the outcome it should influence, the causal story is far more convincing. Sensitivity analyses that vary the model and the window round out the picture.

Reading it in practice

When you meet one of these studies, first identify the counterfactual the authors are leaning on. In interrupted time series it is the extended pre-intervention trend, so ask whether that trend was stable and whether anything else changed at the same moment. In difference-in-differences it is the comparison group, so ask why that group is a fair stand-in and whether the pre-period trends really ran parallel.

Done well, these designs deliver some of the most useful causal evidence available when a randomized trial is off the table. Their honesty is visible in the graphs: a clean interrupted series shows the pre-trend and the projection, and a clean difference-in-differences shows both groups over time. If those pictures are missing and you are handed only a single summary estimate, the assumptions are being asked to carry more weight than they have earned.