Bayesian A/B Test Calculator
Enter visitors and conversions for two variants. The Beta-binomial model gives the probability that B is better than A, the likely size of the uplift, and the expected loss of choosing each variant.
Updated · By Bibin XavierData 90 × 12
| IDtext · 90 levels | Grouptext · 3 levels | Gendertext · 2 levels | Agenumeric | Hours_Studynumeric | Anxietynumeric | Motivationnumeric | Pre_Scorenumeric | Post_Scorenumeric | Exam_Scorenumeric | Passednumeric · coded | Satisfactionnumeric · coded |
|---|---|---|---|---|---|---|---|---|---|---|---|
| S001 | Control | Male | 23 | 11.2 | 29 | 4.6 | 60 | 62 | 67 | 1 | 2 |
| S002 | Workshop | Female | 23 | 3.9 | 28 | 3.5 | 54 | 70 | 54 | 0 | 2 |
| S003 | Online | Female | 18 | 10.1 | 32 | 4 | 55 | 57 | 66 | 0 | 3 |
| S004 | Control | Male | 26 | 13.6 | 41 | 3.5 | 52 | 55 | 59 | 0 | 3 |
| S005 | Workshop | Female | 22 | 9.7 | 27 | 3.6 | 56 | 63 | 65 | 0 | 4 |
| S006 | Online | Female | 20 | 7.9 | 32 | 3.8 | 65 | 82 | 65 | 0 | 2 |
| S007 | Control | Female | 20 | 6.3 | 28 | 3 | 72 | 83 | 61 | 0 | 3 |
| S008 | Workshop | Male | 22 | 8 | 29 | 3.1 | 64 | 72 | 62 | 1 | 3 |
Variables and options
Results
Calculated in your browserPosterior conversion rates
| Variant | Visitors | Conversions | Observed rate | Posterior mean | 95% credible interval |
|---|---|---|---|---|---|
| A (control) | 1,200 | 96 | 8.00% | 8.07% | [6.60%, 9.67%] |
| B (variant) | 1,180 | 121 | 10.25% | 10.32% | [8.65%, 12.12%] |
Note. Beta(1, 1) prior for each rate; posterior Beta(α + conversions, β + non-conversions).
Decision summary
| Quantity | Value |
|---|---|
| P(B > A) | 97.18% |
| P(A > B) | 2.82% |
| Expected relative uplift of B | 29.05% |
| 95% credible interval for uplift | [-0.78%, 65.09%] |
| Expected loss if you choose A | 2.259 percentage points |
| Expected loss if you choose B | 0.013 percentage points |
Note. P(B > A) by numerical integration; uplift and expected loss from 60,000 posterior draws. Expected loss is the average conversion rate you give up if the chosen variant is actually worse.
Posterior distributions of the conversion rates
Distribution of the relative uplift (B vs A)
Interpretation
There is a 97.2% probability that B converts better than A. The expected relative uplift is 29.0% (95% credible interval -0.8% to 65.1%). B is very likely the better variant.
How to cite this tool
Using these results in a thesis, journal article, report or class assignment? Please cite this free tool so others can find it. It keeps the site free.
APA 7
Xavier, B. (2026). Bayesian A/B Test Calculator [Web application]. FreeStatisticalTools.com. https://freestatisticaltools.com/bayesian-ab-test-calculator/
BibTeX
@software{xavier_bayesianabtest_2026,
author = {Xavier, Bibin},
title = {Bayesian A/B Test Calculator},
publisher = {FreeStatisticalTools.com},
year = {2026},
url = {https://freestatisticaltools.com/bayesian-ab-test-calculator/},
note = {Web application}
}Suggested sentence for your methods section: “The analysis was carried out with the free Bayesian A/B Test Calculator tool on FreeStatisticalTools.com (Xavier, 2026).”
What is a Bayesian A/B test?
A Bayesian A/B test treats each conversion rate as uncertain and describes it with a Beta posterior distribution. From these posteriors it calculates the probability that the variant beats the control, the likely size of the uplift and the expected loss of choosing each option, which map directly onto business decisions.
When to use a Bayesian A/B test
Use it to evaluate website, email or product experiments with a binary outcome (converted or not), when you want direct probabilities such as “B is better with 97% probability” rather than a p-value.
Assumptions
- Visitors are randomly assigned and counted once; conversions are independent.
- A flat Beta(1, 1) prior is neutral; informative priors can encode past conversion rates.
How to set up your data
Four numbers: visitors and conversions for each variant.
How to use this calculator
- Enter the values. Fill in visitors a (control), conversions a, visitors b (variant) and conversions b.
- Read the results. Results update automatically. Read the plain-English interpretation, then use Copy table to paste APA-style tables into Word and Copy for the results sentence.
How to report a Bayesian A/B test in APA style
“Variant B converted at 10.3% vs 8.0% for A; the posterior probability that B is better was 98.6%, with an expected uplift of 28% (95% CrI 3% to 57%).”
Frequently asked questions
How do I read “probability to beat control”?
P(B > A) = 95% means that, given the data and prior, there is a 95% chance that B’s true conversion rate is higher than A’s.
When should I stop an A/B test?
Common rules are to stop when P(B > A) exceeds 95% (or falls below 5%) or when the expected loss of the leading variant drops below a threshold you can accept, after a pre-planned minimum sample.
What prior should I use?
Beta(1, 1) is flat and neutral. To encode past knowledge, use α and β proportional to previous conversions and non-conversions.
Is this better than a frequentist significance test?
It answers a more direct question, how likely B is to be better and by how much, and the expected loss helps decide even when the result is not conclusive.
Related tools
Browse all categories
Data cleaning and preparation · Describe and explore · Compare means · Nonparametric tests · Correlation and regression · Bayesian statistics · Categorical data · Reliability and agreement · Multivariate analysis · Structural equation modeling · Econometrics and time series · Panel data and causal inference · Marketing and decision research · Survival, clinical and meta-analysis · Study planning and calculators · PLS-SEM