Leaderboards that survive their own sample sizes.
Your best rep closed three deals.
Every leaderboard ranked by a raw average puts whoever has the fewest observations at the top and the bottom. Ranked shrinks each estimate by exactly how much its sample size deserves, and tells you which units are genuinely different.
Forty sales reps with true close rates injected into the data, scored both ways against the truth. Everything computed in your browser.
The problem
The top of a raw leaderboard is a sample-size ranking wearing a performance costume.
A vendor with two five-star ratings outranks one with four hundred at 4.7. A store with eleven transactions posts the best conversion rate in the chain. A prompt that ran nine times looks like the best prompt you have. Then somebody acts on it: they promote, they investigate, they roll out the winner, they open a performance plan on the unit at the bottom. The ranking was mostly sampling noise, so the actions are mostly noise too, and nobody finds out because the next quarter produces a fresh set of small-sample winners to explain.
The insight
One of the few free lunches in statistics, and nobody is eating it.
James-Stein is not a heuristic and not a tie-breaker. For three or more groups, shrinking every estimate toward the population mean in proportion to how noisy it is beats the raw estimates in total squared error. Always, not usually. The amount of shrinkage is not a knob either: it comes from a prior fitted to the spread of the group itself, after subtracting the sampling noise that spread contains. A unit with ten thousand observations barely moves. A unit with three moves nearly all the way to the mean, which is the correct amount of belief to place in three observations. The alternative most teams reach for, a minimum sample size cutoff, throws those units away instead of using them, and the cutoff is a guess where the prior strength is an estimate.
Beta-binomial empirical Bayes: prior fitted by moments from the observed spread with the binomial sampling variance subtracted first, then a posterior mean and interval per unit. Ranking on the posterior rather than the raw rate. Benjamini-Hochberg across the whole roster, because screening a leaderboard for outliers is one simultaneous test per unit and forty units at p < 0.05 gives you two flags before anybody has done anything unusual.
How it works
Four steps, no data science team
Closed and opportunities, conversions and sessions, ratings and reviews. The denominator is the part every leaderboard drops, and it is the part that decides the ranking.
How much to shrink comes from how much genuine spread the group has once sampling noise is removed. A tight population shrinks hard; a genuinely varied one barely shrinks at all.
Corrected order, the rank each unit moved from, and a credible interval per unit so the ones that are simply unresolved read as unresolved.
FDR-controlled tests against the population rate, so the shortlist of units genuinely worth investigating is a shortlist and not a fishing expedition.
Who it is for
The RevOps or analytics owner whose leaderboard people act on
Anyone who publishes a ranking that people act on: sales leaderboards, store or branch scorecards, supplier and vendor quality tables, creative and prompt-variant boards.
Pricing
- –Shrinkage and interval calculator
- –CSV in, ranked table out
- –Public, no account
- –Scheduled corrected rankings
- –FDR-controlled outlier screening
- –Segment-aware priors
- –Rank-change history over time
- –Self-hosted
- –Warehouse-native models
- –SSO and audit log
- –Review with a statistician
Competition
What exists, and what it does not do
| Who | What they do | The gap |
|---|---|---|
| Salesforce, HubSpot, Looker, Tableau | Where the leaderboard actually lives today. Sort by a computed rate. | Sorting is the whole feature. None of them adjusts for denominator, and the best they offer is a minimum-sample filter you set by hand. |
| Gong, Clari, and revenue intelligence | Deep behavioural analytics on how reps sell. | Rich on the inputs, still ranked on raw outcome rates. The small-sample problem is untouched. |
| A minimum sample size cutoff | The DIY fix: hide anyone under 30 observations. | Discards the units instead of using them, and the cutoff is arbitrary. Shrinkage keeps every unit and weights it correctly, which is strictly more information. |
| A data scientist with a notebook | Exactly right, and what a mature analytics team already does once. | Once. It does not survive into the recurring dashboard that the org actually reads every Monday. |
The likeliest failure is political rather than technical. The person who owns the leaderboard often also owns the rep-of-the-quarter programme that the correction demotes, and "the math punished my best closer" is an easy thing to say and a hard thing to answer in a QBR. The real technical risk is exchangeability: empirical Bayes assumes the units are draws from one population, and a board mixing enterprise and SMB territories is not one population. Shrink across segments that are genuinely different and you will pull a real enterprise star toward an SMB mean and be confidently wrong, which is worse than the raw board it replaced.
Market
Attached to the sales and business-analytics budget, in every company that ranks anything
Every company with more than a handful of comparable units publishes a ranking, and most of them are the same three shapes: reps, locations, suppliers. Teams already pay for the BI seat this sits next to. Four thousand teams on the Team tier is roughly $17M ARR, and a single enterprise with hundreds of stores buys the Scale tier on the store scorecard alone.