A/B test statistical significance calculator
Paste each variant's visitors and conversions to get the uplift, the p-value, and whether the result is real or noise.
Test results
Control (A)
Variant (B)
Chance variant B beats control (Spar)
99.5%
Spar's Bayesian read on a +24% relative uplift. The old frequentist verdict is significant at 95% (p = 0.0101).
Control rate
2.5%
Variant rate
3.1%
Relative uplift
+24%
Frequentist p-value
0.0101
How significance is calculated
This calculator runs a two-proportion z-test, the standard check for whether two conversion rates differ by more than chance. It pools the two variants to estimate the shared rate, measures how many standard errors apart the observed rates are, and converts that into a two-sided p-value.
The result is significant when the p-value falls below your threshold (0.05 for 95% confidence). A significant result means the difference is probably real; it does not, by itself, mean the lift is large enough to be worth shipping.
Read significance alongside effect size
Significance and size answer different questions. The p-value tells you whether a difference is real, and the uplift tells you whether it matters. Ship a change when both line up: a real difference that is also big enough to move revenue. Before you trust a flat result, confirm the test was powered to detect a meaningful lift, because an underpowered test that ends inconclusive is not the same as a tie.
Frequently asked questions
More free CRO tools
Sample size calculator
How many visitors per variant you need to detect a given lift at your chosen confidence and power.
OpenTest duration calculator
How many days a test should run, given your traffic, baseline rate, and the lift you want to detect.
OpenMDE calculator
The smallest lift a test can reliably detect, given your traffic and how long you can run it.
OpenConversion rate calculator
Visitors, conversions, and revenue in; conversion rate, average order value, and revenue per visitor out.
OpenRevenue per visitor calculator
Revenue per visitor from revenue and traffic, plus how a lift in RPV flows through to total revenue.
OpenStop guessing what to test
A calculator tells you whether a test is worth running. Spar finds the tests worth running: it audits your storefront, ranks the gaps by revenue impact, and drafts the variations. Start a 14-day free trial.