Free SRM checker
Is your A/B test split broken?
Enter the visitors each variant got and the split you set. A chi-square test tells you if the difference is normal chance or a sample ratio mismatch that makes the test unreliable.
Verdict
–
- Observed split
- –
- Expected split
- –
- Chi-square
- –
- p-value
- –
Free and private: everything is calculated in your browser. Need someone to plan, design and read your tests? See my A/B testing services.
What sample ratio mismatch means
A sample ratio mismatch (SRM) means your variants got a different share of traffic than you set, by more than chance can explain. If you set 50/50 and got 52/48 on a large sample, something in the setup is biased: a redirect, a bot filter, a tracking bug or a variant that loads slower. Those problems usually also bias the result, so a test with SRM can’t be trusted.
The checker uses Pearson’s chi-square goodness-of-fit test and flags SRM when the p-value is below 0.01, a common, conservative threshold that avoids false alarms.
What to do if you find SRM
Don’t read the result. Find the cause first: check redirects, bot filtering, when users are assigned, and whether one variant fails to fire tracking. I walk through the causes and a worked example in sample ratio mismatch: what it is and how to check.
FAQ
Frequently asked questions
What p-value indicates SRM?
Most teams use p < 0.01, or stricter, as the alarm. A small imbalance is normal; a p-value this low means the split is very unlikely to be chance.
Can I check SRM during the test?
Yes, and you should, early and often. Catching SRM in the first days saves weeks of a broken test.
Does SRM matter if the split is only slightly off?
On a large sample, even a small imbalance can be a real problem. The chi-square test accounts for sample size, so trust the p-value, not your eye.
Should I use visitors or users?
Use the unit your test assigns: usually users or visitors, not page views or sessions.
More free tools
Other A/B testing calculators
CRO ROI calculator
See what a conversion lift is worth per month and per year for your store or lead gen site.
A/B test sample size calculator
How many visitors and how many days your test needs before you can trust the result.
A/B test significance calculator
Is your winner real or just noise? Get the p-value, the lift and its confidence interval.
Let’s talk
Let’s make your next test a winner.
Tell me about your funnel. I’ll point out quick wins, tell you what I’d test first, and say honestly whether I’m the right fit.