A/B Test Statistical Significance, Sample Size & Confidence Calculator
Evaluate conversion rate split tests with mathematical rigor. Calculate required pre-test sample sizes, two-tailed Z-scores, p-values, standard errors, and confidence intervals to eliminate false positives and stop tests with statistical certainty.Documentation & FAQs ↓
Variant B delivered a 17.07% relative lift with 99.17% certainty. It is safe to deploy Variant B.
Step-by-Step Instructions
4 Steps to CompletionFor planning: Enter baseline conversion rate, target Minimum Detectable Effect (MDE), and desired statistical power (typically 80%).
For evaluation: Enter visitors (sample size) and conversions for both Control (A) and Variation (B).
Select your confidence threshold (90%, 95%, or 99%).
Review the two-tailed p-value, Z-score, relative lift percentage, and clear recommendation on whether the test is statistically decisive.
Practical Use Cases & Real-World Scenarios
Pre-Test Sample Size Sizing
Computes minimum sample size per variant based on baseline conversion rate, Minimum Detectable Effect (MDE), statistical power (80%/90%), and alpha significance (95%/99%).
Two-Tailed Z-Score & P-Value Engine
Evaluates post-test results using two-proportion pooled Z-tests, calculating exact p-values to prove whether conversion differences are statistically genuine.
Confidence Interval & Uplift Range
Calculates 90%, 95%, and 99% confidence intervals for absolute and relative conversion lift, modeling worst-case and best-case revenue outcomes.
Early-Stopping & Peeking Hazard Warnings
Flags false-positive risks associated with repeated data peeking and recommends sample completion thresholds before declaring winners.
Frequently Asked Questions
What does a 95% confidence level (p < 0.05) actually mean in A/B testing?↓
Why is stopping an A/B test early ('peeking') dangerous?↓
What is Statistical Power (1 - Beta) and why does 80% matter?↓
What is the difference between absolute lift and relative lift?↓
Can this calculator evaluate tests with more than two variations (A/B/C/n)?↓
15 Best Data Analytics Tools for Analysis in 2026
Dive deeper into the foundational technical concepts behind this tool on our editorial blog. Includes comprehensive hardware benchmarks, case studies, and best practices.
Found this tool helpful? Share it with colleagues & friends:
100% free, private in-browser utility with zero server uploads.