Synthetic Control Method

The idea behind synthetic control

Many interventions affect only one unit: a campaign is tested in one region, a new store format opens in one city, a law changes in one state, a price change applies to one country. Comparing the treated unit with a single other unit is unconvincing, because no single unit is a perfect match. The synthetic control method, developed by Alberto Abadie and colleagues, builds a better comparison: a weighted average of untreated units (the donor pool) chosen so that it reproduces the treated unit’s outcome as closely as possible before the intervention. After the intervention, this synthetic control shows what would probably have happened to the treated unit without it, and the gap between the two is the estimated effect. Athey and Imbens (2017) called it “arguably the most important innovation in the policy evaluation literature in the last 15 years”, and it is now widely used for geo-experiments in marketing.

How the weights are found

The tool finds donor weights w that minimise the squared differences between the treated unit’s outcome and the weighted donor outcomes over all pre-intervention periods, subject to two constraints: all weights are non-negative and they add up to one. These constraints prevent extrapolation (the synthetic unit is a genuine convex combination of real units) and usually produce a sparse solution in which only a few donors get positive weight, which makes the comparison transparent. The problem is a quadratic programme on the simplex, solved here with an accelerated projected-gradient method; the weights match a standard constrained optimiser (SLSQP) to three decimals.

This implementation matches on the pre-intervention outcome path, the most common and robust specification in practice. Abadie’s original method can also match on covariates such as income or population; when the pre-period is long, matching on outcomes alone captures most of what covariates would add.

Measuring fit and effect

• Pre-period RMSPE (root mean squared prediction error) measures how well the synthetic control tracks the treated unit before the intervention. Express it relative to the outcome’s level: a fit within a few percent is good. If the treated unit cannot be reproduced (for example because it is the largest or most extreme unit), the method is not reliable.
• Gap: treated minus synthetic in each period. The average post-intervention gap is the estimated average treatment effect on the treated unit; the tool also expresses it relative to the synthetic outcome.
• Post/pre RMSPE ratio: how much larger the gaps are after the intervention than before. A large ratio means a clear break.

Placebo inference

With one treated unit, standard errors are not available. Instead, synthetic control uses placebo tests in space: each donor is treated in turn as if it had received the intervention, and a synthetic control is built for it from the remaining donors. If the real treated unit’s post/pre RMSPE ratio is larger than most placebo ratios, the effect is unlikely to be a chance result. The p-value is the share of all units (treated plus placebos) with a ratio at least as large as the treated unit’s. With 16 units, the smallest possible p-value is 1/16 = .0625. The gap chart shows the placebo gaps in grey; placebos with a much worse pre-period fit than the treated unit are hidden because their post-period gaps are not informative.

Worked example with the built-in data

The example contains yearly sales per 1,000 inhabitants for 16 regions from 2010 to 2023; a regional campaign started in the North in 2018. The synthetic North combines East (weight .33), Lakes (.24), Valley (.18), Coast (.14) and South (.11). Before 2018 it tracks the North very closely (RMSPE 1.17, under 1% of the mean). From 2018, the North pulls away: by 2023 it is 18.6 above its synthetic control, and the average post-campaign effect is +11.4 (7.5%). The post/pre RMSPE ratio of 10.6 is the largest of all 16 units, giving the smallest possible placebo p-value of .063: no untreated region shows a comparable break.

Data and design requirements

• Long panel data: one row per unit and period with the unit name, the period and the outcome. Every donor should have all periods; incomplete donors are dropped.
• Many pre-intervention periods: ideally 10 or more, so that a good fit reflects genuine similarity rather than overfitting a few points.
• A clean donor pool: exclude units that were also treated, affected by spillovers (neighbouring regions reached by the same media), or hit by their own large shocks.
• Comparable units: donors should be structurally similar to the treated unit. The treated unit must lie within the range of donor outcomes, since weights cannot extrapolate.
• Enough donors for meaningful placebo inference: with fewer than about 10, the smallest achievable p-value is large.

Synthetic control in marketing

In marketing, synthetic control is most often used for geo-experiments and matched-market tests: a campaign, a new channel, a price change or a store concept is launched in one or a few markets, and its effect on sales, sign-ups or traffic is measured against a synthetic version built from the other markets. It is also used to evaluate events that were not planned as experiments, such as a competitor’s exit from one region. Compared with difference-in-differences, it relaxes the parallel-trends assumption by constructing a comparison that matches the treated unit’s own trajectory.

Limitations

• A good pre-period fit is necessary but not sufficient; an unobserved shock to the treated unit at the same time as the intervention would still bias the estimate.
• With few post-periods or noisy outcomes, effects are hard to distinguish from noise; the placebo distribution shows this.
• For several treated units, run the analysis for each or use difference-in-differences or augmented and generalised synthetic control methods.

Reporting

Report the treated unit, the treatment date, the donor pool, the weights, the pre-period fit, the average post-intervention effect and the placebo p-value, with the treated-versus-synthetic chart. Example: “A synthetic control for the North region was constructed from 15 donor regions (pre-treatment RMSPE = 1.168). After the 2018 campaign, sales were on average 11.36 per 1,000 above the synthetic control (7.5%); in-space placebo tests gave p = .063, the lowest attainable with 16 units.”

Glossary of synthetic control terms

• Treated unit: the region, store or country that received the intervention.
• Donor pool: the untreated units available to build the synthetic control.
• Convex combination: a weighted average with non-negative weights that sum to one, so the synthetic unit lies within the range of donors.
• Pre-treatment fit: how closely the synthetic control matches the treated unit before the intervention.
• Gap: the difference between the treated unit and its synthetic control in a period.
• In-space placebo: applying the method to each untreated unit as if it were treated, to build a reference distribution.
• In-time placebo: pretending the intervention happened earlier to check that no effect appears before the true date.
• Spillover: effects of the intervention on donor units, which bias the estimate.

Key references: Abadie and Gardeazabal (2003), American Economic Review; Abadie, Diamond and Hainmueller (2010), Journal of the American Statistical Association; Abadie (2021), Journal of Economic Literature.