Free A/B test significance calculator
Is your A/B test winner real?
Enter visitors and conversions for the control and the variant. You get the conversion rates, the lift, the p-value, a confidence interval and a plain-English verdict.
Verdict
–
- Conversion rate A
- –
- Conversion rate B
- –
- Relative lift
- –
- p-value (two-sided)
- –
- Lift confidence interval
- –
Warning: the visitor counts look like a sample ratio mismatch for a 50/50 test. Check the split before you trust this result.
Get help running your testsFree and private: everything is calculated in your browser. Need someone to plan, design and read your tests? See my A/B testing services.
How to read the result
A significant result means the difference is unlikely to be pure chance, not that the lift is exactly the number you see. The confidence interval shows the range the true lift probably falls in. If it is wide, the real effect could be much smaller than the observed one.
The calculator uses a two-sided two-proportion z-test. It is the right check for conversion rates when the test ran to its planned sample size. It is not a license to stop a test the moment it turns green.
Before you call a winner
- Did the test reach the sample size you planned? Use the sample size calculator first.
- Did it run at least two full weeks?
- Is the traffic split right? Check it with the SRM checker.
- Did revenue per visitor move too, not just clicks? See the revenue impact of an A/B test.
If the result is flat, read what to do when an A/B test is inconclusive.
FAQ
Frequently asked questions
What p-value means significant?
At 95% confidence, a p-value below 0.05. It means a difference this large would be unlikely if there were really no difference. It does not tell you how big the true effect is.
Is this a one-sided or two-sided test?
Two-sided. It checks whether B is better or worse than A, which is the safer default for most tests.
My test is significant after two days. Can I stop?
No. Early results swing a lot. Run to the sample size you planned and at least two full weeks, then read the result once. See when to call an A/B test.
Can I use this for revenue per visitor?
Not directly. This calculator is for conversion rates. Revenue per visitor needs a test on averages, which your testing tool or a t-test on order-level data can do.
More free tools
Other A/B testing calculators
CRO ROI calculator
See what a conversion lift is worth per month and per year for your store or lead gen site.
A/B test sample size calculator
How many visitors and how many days your test needs before you can trust the result.
SRM checker
Check if your traffic split matches what you set. A mismatch means the test can’t be trusted.
Let’s talk
Let’s make your next test a winner.
Tell me about your funnel. I’ll point out quick wins, tell you what I’d test first, and say honestly whether I’m the right fit.