A/B Test Significance
Check if the difference between two variants is significant.
UPLIFT (B vs A)
+30.0%
Significant at 95%
96.5%
Confidence
0.035
p-value
Two-proportion z-test. A p-value below 0.05 (95% confidence) is the usual bar for calling a winner — collect enough data before trusting the result.
What is A/B Test Significance?
A/B test significance tells you whether the difference between two variants is real or just noise. This tool runs a two-proportion z-test and reports the uplift, p-value, and confidence level.
z = (pB − pA) ÷ pooled standard error · significant if p < 0.05
How to read your result
- 95% confidence (p < 0.05) is the common threshold for calling a winner.
- Small samples rarely reach significance — collect enough data first.
- A big uplift can still be insignificant if the sample is tiny.
- Don't stop a test early the moment it looks significant — run it to plan.
Frequently asked questions
What does statistical significance mean in an A/B test?
It means the observed difference between variants is unlikely to be due to random chance. A p-value below 0.05 (95% confidence) is the usual bar for declaring a result significant.