The recommendation
What the product actually hands you
A schedule change with the evidence attached. Not a conversion rate, not a dashboard tile. The output is a day number, because the day is the thing you can change.
Decisive day 3 · hazard 10.5% · 31 conversions on that day alone
Cumulative 81.3% of this flow's conversions land by day 4
Current schedule welcome d0 · activation nudge d7 · expiry sequence d12
Recommended welcome d0 · activation nudge d2 · stalled check d5 · expiry d12
Reason the d7 nudge arrives after the decision has already been made
for 92.3% of the users who will convertThe nudge is not wrong, it is late. On this flow the conversion decision resolves on day 3, which means the day-7 email is being sent almost entirely to users who already converted or already stopped opening. Moving it to day 2 puts it in front of the population that is still deciding. The day-5 stalled check is new: it is the first day where a non-converter is genuinely informative rather than just early.
Decisive day 13 · hazard 8.7% · 86 conversions on that day alone
Cumulative 9.4% of conversions by day 4, 21.1% by day 9
Shape flat hazard through day 9, then a spike at the expiry sequence
Reading the trial deadline is making the decision, not the product
Recommended treat d13 conversions as at-risk revenue and instrument
first-value time separately before touching the email scheduleA hazard curve that is flat for nine days and then spikes at the deadline is the signature of users who never reached the thing the product is for and bought anyway because the clock ran out. That converts, and it churns. This one is not a schedule fix. Sending the expiry email earlier would move the spike, not the outcome.
Completed only 28.3% on 1180 vs 36.0% on 100
z = -1.63, p = 0.10 · cannot conclude
Log-rank, all chi-square = 44.8, p = 2.2e-11
Observed/expected Control onboarding: 351 vs 394 (0.89x)
Guided setup: 91 vs 48 (1.88x)
14-day total 28.2% [25.7% to 30.7%] vs 32.0% [25.8% to 38.1%]
Status timing separated · totals not separatedTwo findings, and only one of them is settled. The conversion hazard of the two flows is different beyond any doubt, which is enough to change the email schedule today. The 14-day totals are not separated: the intervals overlap, and guided setup has only reached day 14 for a hundred trials. Do not call the variant a winner on conversion rate yet. Do move the nudge, because that decision rests on the finding that is resolved.
Trial conversion, as your dashboard currently reports it, is 24.6%. It is actually 30.1%. The difference is 520 trials that have not finished yet. That number is not a finding and it is not actionable, which is exactly why it should not be the headline metric of a trial program.