link
Sample size calculator
Required sample size per variantiMinimum users or sessions needed for each variant.---
One calculator for pre-test planning and post-test analysis.
1
How long has the test been running?
Test duration
Additional days needediIf indicated, the number of additional days that is needed to run the test considering the existing CR, variant lift, and sample size.---
2
Test data
Control
Conversion rateiConversions divided by users or sessions.---
Variation 1
Conversion rateiConversions divided by users or sessions.---
LiftiRelative difference between the variation conversion rate and the control conversion rate.---
Extra monthly transactionsi(CR Original * Monthly Traffic) - (CR Variant * Monthly Traffic)---
Monthly monetary contributioniExtra monthly transactions multiplied by average order value or lead value.---
P-valueiProbability of getting a similar or larger difference when repeating the test.---
SignificanceiWhether the result passes the selected significance threshold.Not significant
SRM-alert: if you intended to have a 50% / 50% split, we measured a possible Sample Ratio Mismatch (SRM).
Please check your traffic distribution.
Please check your traffic distribution.
3
Test statistic details and supporting calculators
link
Duration calculator
How long in total to run the testiEstimated total test duration based on sample size, traffic, and variants.---
Monthly monetary contribution based on data above
Monthly contributioniEstimated monthly monetary impact for the selected variant.---
Minimum detectable effect (MDE) iMinimum relative effect detectable for each weekly test duration.
| Week | MDE |
|---|---|
| 1 | --- |
| 2 | --- |
| 3 | --- |
| 4 | --- |
| 5 | --- |
| 6 | --- |
4
Results
Results
Significance calculation method iChoose Bayesian probability or frequentist significance testing.
Probability of being a winner
% chance of out performing
0.00.20.30.50.70.81.0
Probability density
Conversion Rate (%)