A/B Test Significance Calculator
Enter visitors and conversions for two variants to get the conversion rates, lift, p-value, confidence interval and a clear significant or not significant result.
Enter visitors and conversions for two variants to get the conversion rates, lift, p-value, confidence interval and a clear significant or not significant result.
When variant B converts better than A, is it a real improvement or just chance? A significance test answers that. This calculator runs a two-sided two-proportion z-test on your numbers.
You get both conversion rates, the lift of B over A, a confidence interval for the difference, the z-score and p-value, and a plain-language verdict at the confidence level you choose.
The test pools both variants to estimate a single overall conversion rate under the assumption that there is no difference. It then computes a standard error from that pooled rate and the two sample sizes, and divides the observed gap by it to get a z-score. The p-value is the two-sided probability of a gap at least that large. Example: A converts 100 of 1,000 (10 percent) and B converts 120 of 1,000 (12 percent). The z-score is about 1.43 and the p-value about 0.15, so the result is not significant at 95 percent.
The confidence interval uses a separate, unpooled standard error and shows a plausible range for the true difference in percentage points. In the example it spans from slightly below zero to roughly five points above, which is why a two-point lift cannot be trusted yet. The z-test relies on a normal approximation, so it works best when each variant has at least a few dozen conversions and non-conversions. With very small counts, results are unreliable. It also tests one comparison, not several variants or many metrics at once.
The probability of seeing a difference this large if there were really no difference. A p-value below 0.05 is significant at 95% confidence.
Not safely. Checking repeatedly and stopping early inflates false positives. Decide the sample size first with the Sample Size Calculator.
No. A tiny but significant lift may not be worth the change. Look at the interval as well.
No. The test runs in your browser.
It means the data cannot distinguish the variants from chance. You can run longer if you planned for more traffic, accept that any difference is small, or test a bolder change.
Not directly. It compares exactly two. Running several pairwise tests raises the chance of a false positive, so use a method that corrects for multiple comparisons for larger experiments.