A/B Test Significance Calculator
Find out whether variant B really converts better than A, with the relative uplift, a confidence interval for the difference and the chance that B beats A.
Updated · By Bibin XavierData 10 × 4
| Studytext · 10 levels | Effect_Size_dnumeric | SEnumeric | Nnumeric |
|---|---|---|---|
| Adams 2015 | 0.05 | 0.13 | 238 |
| Bose 2016 | -0.06 | 0.22 | 81 |
| Chen 2017 | 0.31 | 0.12 | 285 |
| Duarte 2018 | 0.18 | 0.15 | 177 |
| Eriksen 2018 | 0.25 | 0.13 | 229 |
| Fofanah 2019 | 0.75 | 0.31 | 41 |
| Gupta 2020 | 0.01 | 0.28 | 51 |
| Halvorsen 2021 | 0.14 | 0.2 | 97 |
Variables and options
Results
Calculated in your browserTest results
| Statistic | Value |
|---|---|
| Difference (B − A) | 1.41 pp |
| 95% CI for difference | [0.28, 2.53] pp |
| z | 2.446 |
| p (two-tailed) | .014 |
| Decision | Significant at 95% confidence |
Conversion rate with CI
Interpretation
Variant B converts significantly better. B's rate is 17.6% higher than A's, and the true difference likely lies between 0.28 and 2.53 percentage points.
How to cite this tool
Using these results in a thesis, journal article, report or class assignment? Please cite this free tool so others can find it. It keeps the site free.
APA 7
Xavier, B. (2026). A/B Test Significance Calculator [Web application]. FreeStatisticalTools.com. https://freestatisticaltools.com/ab-test-significance-calculator/
BibTeX
@software{xavier_abtestcalculator_2026,
author = {Xavier, Bibin},
title = {A/B Test Significance Calculator},
publisher = {FreeStatisticalTools.com},
year = {2026},
url = {https://freestatisticaltools.com/ab-test-significance-calculator/},
note = {Web application}
}Suggested sentence for your methods section: “The analysis was carried out with the free A/B Test Significance Calculator tool on FreeStatisticalTools.com (Xavier, 2026).”
What is an A/B significance test?
Check whether the difference in conversion rate between two variants is statistically significant, how large the uplift is, and how confident you can be that B beats A.
When to use an A/B significance test
Use it to evaluate landing pages, emails or app features where the outcome is a conversion (yes/no). Decide the sample size before starting, using the power analysis tool with two proportions.
How to use this calculator
- Enter the values. Fill in visitors a, conversions a, visitors b and conversions b.
- Read the results. Results update automatically. Read the plain-English interpretation, then use Copy table to paste APA-style tables into Word and Copy for the results sentence.
Frequently asked questions
When is an A/B test result significant?
When the p-value is below 1 minus your confidence level, for example below .05 at 95% confidence.
Can I stop the test as soon as it is significant?
No. Checking repeatedly and stopping early inflates false positives. Decide the sample size in advance with a power analysis.
What is relative uplift?
The percentage change in conversion rate from A to B, (rate B ÷ rate A − 1) × 100.
Related tools
Browse all categories
Data cleaning and preparation · Describe and explore · Compare means · Nonparametric tests · Correlation and regression · Bayesian statistics · Categorical data · Reliability and agreement · Multivariate analysis · Structural equation modeling · Econometrics and time series · Panel data and causal inference · Marketing and decision research · Survival, clinical and meta-analysis · Study planning and calculators · PLS-SEM