Causal readouts for changes you cannot A/B test.
Before/after credited your rollout with a trend that hit every region.
Pricing changes, migrations, regional launches and policy changes cannot be randomised, so teams read a before/after chart and bank the number. Rollout subtracts what happened to the groups you did not touch, and refuses to report at all when the comparison group is not good enough.
A rollout with a known injected effect, a seasonal trend that hit everyone, and two control groups that fail the check.
The problem
The decisions that matter most are the ones you cannot randomise.
You can A/B test a button. You cannot A/B test a price change, an infrastructure migration, a new support tier, a regional launch, or a policy change, because they ship to whole populations at once. So the readout falls back to a before/after chart, which is confounded by every other thing that moved in the same window. The usual outcome is that a secular trend gets credited to the change, the change gets called a win, and the next quarter of strategy is built on it.
The insight
The control group is already sitting in your warehouse.
Almost every non-randomisable change is still a partial rollout. Some regions got it and some did not, or some segments, or some services, or some cohorts. Difference-in-differences subtracts what happened to the untreated group from what happened to the treated one, which cancels anything that moved for both. The assumption it needs is that the two groups were moving in parallel before the change, and that assumption is checkable against data you already have. Checking it is most of the value here, because the failure mode is not a slightly noisy estimate, it is a confident number computed against a group that was never comparable. So the check runs first, and when it fails the product returns a refusal and the reason instead of an answer.
Difference-in-differences on pooled treated versus pooled control, gated on a pre-period parallel-trends correlation with a stated threshold. Every candidate control is scored before an estimate is issued. Backed by an in-time placebo fitted on a window where nothing shipped, and by agreement across each control group taken on its own.
How it works
Four steps, no data science team
The date it went live and the regions, segments or services it went live in. Everything else that is on the same metric and did not get the change becomes a candidate control.
Each candidate control is checked for parallel pre-trends against the treated group. The score, the threshold, and the pass or fail are all shown, so the choice of control is an argument you can have rather than a default you inherit.
Above the threshold you get the effect, the naive number for contrast, and how much of the naive number was trend. Below it you get a refusal naming the control that failed and by how much. Both are useful. Only one of them is an answer.
The same estimator is refitted on the pre-period with a fake rollout date. It should return roughly zero. When it does not, the design is reading structure rather than the change, and the readout says so.
Who it is for
The team that has to say whether the change worked
Product and platform teams that ship changes to whole populations and currently settle the question with a line chart and an argument. Pricing, packaging, infrastructure, regional expansion, trust and safety policy.
Pricing
- –Parallel-trends scoring
- –DiD calculator
- –CSV in, verdict out
- –No account
- –Warehouse sync and metric definitions
- –Automatic control-group search
- –Placebo and sensitivity checks
- –Readouts that link back to the change
- –Self-hosted
- –Synthetic control for single-unit rollouts
- –SSO and audit log
- –Review with a statistician
Competition
What exists, and what it does not do
| Who | What they do | The gap |
|---|---|---|
| Statsig, Eppo, LaunchDarkly | Experimentation platforms with strong randomised analysis and, increasingly, some quasi-experimental support. | Built around the randomised case and the feature flag that produced it. The changes here have no flag and no random assignment, and the hard part is choosing and defending a control group rather than analysing an assigned one. |
| An analyst with a notebook | Exactly right, and what actually happens at companies that can afford it. | One-off, unversioned, and the pre-trend check is the step most likely to be skipped when the answer is already expected. Also does not scale past the analyst. |
| BI dashboards | Where the before/after chart lives today. | Shows the treated line. Nothing in the tool suggests you should be subtracting a control line from it. |
| Media mix and marketing causal tools | Causal inference sold to marketing. | Same maths, different buyer and different data. Nobody is selling this to the platform team shipping a migration. |
The failure mode is that the product is right and unwanted. A team that already shipped the change and told leadership it worked does not want a tool whose main output is sometimes "we cannot tell you". The buyer has to value being correct over being able to claim a win, and that is a real filter on the market. The counter is to sell it before the rollout, as part of designing it, when nobody has a number to defend yet.
Market
Attached to the experimentation budget, sold to teams that cannot use experimentation
Companies already pay $1,000 to $10,000 a month for experimentation platforms covering the randomisable minority of their decisions. This is the same buyer and a smaller line item for the decisions those platforms cannot touch. Three thousand teams at the Team tier is $32M ARR, and the Scale tier is the natural upsell once more than one team is shipping non-randomisable changes.