Formula
p̂ = (c_A + c_B) ÷ (n_A + n_B)SE = √(p̂(1 − p̂)(1/n_A + 1/n_B))z = (p_B − p_A) ÷ SEUplift = (p_B − p_A) ÷ p_A How it works
An A/B test shows two versions of a page, email or feature to separate groups of users and compares how often each group converts. Because some difference will always appear by chance, you need a statistical test to judge whether the gap is real. This calculator uses the standard two-proportion z-test.
The test pools both groups to estimate the conversion rate you would expect if the variants were identical, uses it to compute the standard error, and then measures how many standard errors apart the two observed rates are. That z-score is turned into a p-value. If the p-value is below 1 minus your confidence level (0.05 for 95%), the result is called statistically significant.
The relative uplift is the headline number marketers quote: a jump from 8% to 9.4% is a 17.5% relative uplift, even though the absolute difference is 1.4 percentage points. The confidence interval for the difference shows the range of effects that are consistent with your data.
Decide on the sample size before you start and avoid stopping the test the moment it turns significant — peeking repeatedly inflates the false-positive rate well above the nominal 5%.
Frequently asked questions
Should I use a one-sided or two-sided test?
Two-sided is the safer default: it detects a difference in either direction. Use one-sided only if you decided in advance that you only care whether B beats A, and would treat “B is worse” the same as “no difference”.
How many visitors do I need?
It depends on your baseline rate and the smallest uplift you want to detect. Small effects on low conversion rates can require tens of thousands of visitors per variant. Plan with a power analysis before launching.
What does 95% confidence mean here?
It means you accept a 5% chance of declaring a winner when there is actually no difference (a false positive).