A/B Test Sample Size Calculator

Why sample size matters in A/B testing

Ending an A/B test too early is the most common way to get a wrong answer. With too few visitors, random noise can look like a winner & a real improvement can look like nothing. This calculator tells you, before you launch, how many visitors each variant needs so that a genuine lift is likely to show up as statistically significant.

It uses the standard proportion formula n = 16 × p(1−p) / δ², a widely used approximation for tests run at 95% confidence and 80% power. Here p is your baseline conversion rate and δ is the absolute lift you want to be able to detect. Everything runs in your browser — nothing you type is uploaded or stored. Results are estimates only, not professional advice; real-world traffic and conversion behavior vary.

How to use it

1. Enter your baseline conversion rate — the current rate of the page or flow you are testing (for example 3%).
2. Enter the minimum detectable effect: the smallest lift worth acting on. Choose relative (a 20% relative lift on a 3% baseline means reaching 3.6%) or absolute (percentage points added directly).
3. Set the number of variants, including your control. Most tests use 2.
4. Optionally add your daily visitors to see roughly how long the test will take.
5. Read the results instantly: sample per variant, total sample & the conversion rate you would be able to detect.

FAQ

What do 95% confidence and 80% power mean?

95% confidence limits false positives: if there is truly no difference, you will wrongly declare a winner only about 5% of the time. 80% power limits false negatives: if the true lift equals your minimum detectable effect, the test will catch it about 80% of the time.

Should I use a relative or absolute effect?

Relative is more common in marketing ("we want at least a 10% lift"). Absolute is useful when you think in percentage points, such as moving 3.0% to 3.5%. Both produce the same math once converted to an absolute delta.

Why does a smaller effect need so many more visitors?

Sample size grows with the square of the effect. Halving the effect you want to detect roughly quadruples the visitors required, which is why detecting tiny lifts on low-traffic pages is often impractical.

Can I stop the test early if one variant is clearly winning?

Peeking and stopping early inflates false positives. Decide the sample size up front, run to completion & only then read the result — or use a sequential testing method designed for early stopping.