When should I use Z-test vs Chi-Square for A/B testing?

For 2 groups with binary outcomes, both give identical results (chi-square = z-squared). Z-test also provides confidence intervals. Chi-Square supports 3+ groups.

When should I use Fisher's exact test?

Use Fisher's exact test when sample sizes are small and expected cell counts are below 5, where chi-square approximation is unreliable.

What confidence level should I use for my A/B test?

95% is the industry standard. Use 90% for exploratory tests, 99% for high-stakes decisions.

Conversions A/B test calculator

Question: Is there a statistically significant difference between my A/B test conversion rates?

Analyze your A/B test conversion results with three switchable statistical methods: Z-test for proportions, Chi-Square test, and Fisher Exact test. Enter visitors and conversions for each variant to get instant results with p-values, confidence intervals, and distribution visualizations.

How to use this calculator

Enter the number of visitors and conversions for your control group (A) and variant group (B). Select your desired confidence level (95% is the industry standard) and whether you want a one-tailed or two-tailed test. Results update in real-time as you type — no need to click a button.

How the math works

This calculator offers three methods for comparing conversion rates. The Z-test uses the two-proportion z-test, calculating the difference in proportions divided by the pooled standard error. The Chi-Square test compares observed vs expected frequencies in a contingency table. Fisher Exact test computes the exact probability for small samples where approximations may be unreliable. All three methods test whether the difference in conversion rates is statistically significant.

When to use this calculator

Use this calculator after running an A/B test to determine if the observed difference in conversion rates is statistically significant or could be due to random chance. This is the most common analysis for marketing experiments, product feature tests, and UX optimization. It works best when you have binary outcomes like clicks, sign-ups, or purchases.

Common mistakes in A/B testing

The most common mistakes include peeking at results before reaching the required sample size (which inflates false positive rates), running underpowered tests that cannot detect meaningful differences, not accounting for multiple comparisons when testing more than two variants, and stopping tests too early based on initial trends. Always determine your sample size before starting, and let the test run to completion.