Live demo
16 series in. 4 alerts out.
A consumer subscription business, 60 days of daily cohorts, 16 series across four acquisition channels. On day 41 one channel started sending a different kind of user. Blended month-3 retention, the number in the board deck, never moved. Find the channel and find the region of the curve.
Every number on this page is computed in your browser right now, by the same CUSUM and Bayesian changepoint code that runs in production. Nothing here is a screenshot.
The comparison that matters
Against the rule most teams actually ship
The standard configuration is a percentage or sigma threshold against a trailing mean, evaluated per metric. Run it over the exact same 16 series and count what lands in the channel.
The false-alarm count is not a judgement call. The incident was injected at a known index, so every naive alert before that index is wrong by construction.
What the engine found
Detection run
| Metric | Trend | Baseline | Now | Change | Confidence | Method |
|---|---|---|---|---|---|---|
Hours to second session channel=paid_social | 29.022 | 56.574 | +94.9% | 99.9% | CUSUM | |
Week-2 retention channel=paid_social | 41.1% | 25.8% | −37.3% | 99.9% | CUSUM | |
Sessions per active user channel=paid_social | 4.744 | 3.144 | −33.7% | 99.9% | CUSUM | |
Day-7 activation rate channel=paid_social | 56.0% | 39.8% | −29.0% | 99.9% | CUSUM |
Show your work
Hours to second session · channel=paid_social
Baseline mean μ = 0.0039 and σ = 0.0635, both computed from the first two thirds of the day-over-day percentage changes. The slack k = 0.0318 is half a sigma, and the decision boundary h = 0.2542 is four. The accumulator runs on percentage changes rather than raw levels so a healthy growing series cannot drift across the boundary on its own.
alert if |Δ| > 2.5σ vs 7-day mean fired 5 times across this one metric, including 1 before anything was actually wrong. The statistical pipeline sent one message, on the day the regime actually changed.
What lands in Slack
One message, with the work already done
- 1.Diff the paid_social landing page and creative set against the Aug 8 version.
- 2.Compare first-session event streams for cohorts on either side of Aug 9.
- 3.Check whether the new creative changed the promise the landing page pays off.
The written cause is generated only after the statistics confirm the change. The model never decides whether something is an anomaly — it explains one that has already been established. Getting that order backwards is how these products hallucinate.