Ground Truth.
AI, checked against the source.

Learn · Intermediate

Staggered difference-in-differences: learning from changes that happen at different times

Staggered difference-in-differences is a way to estimate the effect of an intervention when different groups adopt it at different times. It compares changes in earlier adopters with changes in groups that have not yet adopted, using the latter to approximate what might have happened without adoption. The method can make observational data more informative, but its conclusion depends on assumptions about those missing alternatives.

Imagine several engineering teams introduce coding agents in different months. Pull requests might rise after adoption, but that alone is not proof the agents caused the increase: a company-wide product launch could have increased development work everywhere. Comparing the early adopters’ before-and-after change with the change among teams that have not yet adopted can subtract a shared trend. That is the basic intuition behind difference-in-differences.

Suppose early adopters average ten merged changes per month before adoption and fourteen afterward. Over the same periods, comparable later adopters go from ten to twelve. The simple difference-in-differences estimate is (14 − 10) − (12 − 10), or two additional changes. The control group’s increase accounts for a common two-change rise. This illustration is deliberately simple: it does not prove the groups were comparable or describe a particular study.

The central assumption is parallel trends. Without the intervention, the treated and comparison groups would have changed similarly on average. They do not need identical baseline levels, but their untreated trends need to be sufficiently comparable for the question. The missing outcome—what actually would have happened to early adopters had they waited—can never be observed directly. Researchers instead defend the assumption with knowledge of adoption, comparison groups and evidence from periods before adoption.

Pre-adoption trends help, but they are not a certificate. Similar past trends can coexist with a future shock that affects adopters differently. Conversely, noisy data can conceal an existing divergence. A good study explains why adoption happened when it did, looks for relevant changes in the surrounding organization, and reports uncertainty. The site's causal graphs lesson explains why deciding what to compare is a substantive question rather than a purely statistical one.

Staggering introduces an extra complication: a firm that adopts in January is already treated when a second firm adopts in June. Traditional two-way fixed-effects regressions can inadvertently use earlier-treated groups as controls for later-treated groups. If effects change over time or differ by adoption cohort, the resulting average can be difficult to interpret and can even receive counterintuitive weights. Andrew Goodman-Bacon’s analysis of variation in treatment timing explains how the conventional estimator combines comparisons.

Modern approaches make the comparison explicit. Brantly Callaway and Pedro Sant’Anna’s group-time treatment-effects framework estimates effects for a cohort at a particular time, using never-treated or not-yet-treated groups as appropriate controls, and then combines those estimates with stated weights. Liyang Sun and Sarah Abraham’s work on event studies shows why apparent lead and lag effects from conventional regressions can be contaminated when treatment effects differ across cohorts. These methods address important weighting problems, but they do not conjure a valid control group if the identifying assumptions fail.

An event-study graph places time relative to adoption on the horizontal axis. It can show whether an estimated effect appears immediately, grows or fades. Its points before adoption are often used as a diagnostic for divergent trends or anticipation. But a quiet pre-period is not proof of parallel trends, and many separate estimates can be noisy. Confidence intervals and sample sizes matter, especially far from the adoption date when few cohorts may contribute.

The Chen and Stratton study of coding-agent adoption illustrates the interpretive stakes. Its firms adopted at different times; the authors compare changes around adoption and report a longer elapsed pull-request review process. That is evidence about a possible organizational bottleneck under the study’s assumptions. It does not directly identify whether more incoming code, code quality, management choices or another accompanying change explains the delay.

When reading such a result, ask four questions: who adopted and when; which groups provided the counterfactual at each time; whether effects vary across cohorts or time since adoption; and what other changes could track adoption. A randomized trial can provide stronger protection against selection when feasible, but even a trial measures an effect for its own participants and conditions. Staggered designs are valuable precisely because many real interventions cannot be randomly scheduled; their credibility comes from transparent comparisons, not from the name of the method.

Key papers
Goodman-Bacon: Difference-in-differences with variation in treatment timing
Callaway and Sant’Anna: Difference-in-differences with multiple time periods
Sun and Abraham: Estimating dynamic treatment effects in event studies

Key questions

What does staggered difference-in-differences compare?

It compares changes in groups adopting an intervention at different times, using never-treated or not-yet-treated groups to estimate a counterfactual.

Does a flat pre-adoption trend prove the estimate is causal?

No. Similar past trends support the design but cannot prove that untreated groups would have followed parallel trends afterward.

Why can ordinary fixed-effects estimates be misleading with staggered adoption?

Already-treated groups may serve as controls for later adopters, producing hard-to-interpret averages when effects vary by cohort or time.
Cite this

APA

Ground Truth. (2026, November 18). Staggered difference-in-differences: learning from changes that happen at different times. Ground Truth. https://groundtruth.day/learn/staggered-difference-in-differences.html

BibTeX

@misc{groundtruth:staggered-difference-in-differences,
  title  = {Staggered difference-in-differences: learning from changes that happen at different times},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {nov},
  url    = {https://groundtruth.day/learn/staggered-difference-in-differences.html}
}

Topics: causal-inference · evaluation · statistics