What is a ratio metric in A/B testing?

A ratio metric is computed as sum(X)/sum(Y) — like AOV (revenue/orders), ARPU (revenue/users), or CTR (clicks/impressions).

Why not just use a t-test on ratio metrics?

Simple averaging of per-user ratios is biased when users contribute different amounts. The delta method provides unbiased variance estimation.

What is the delta method?

The delta method approximates the variance of a ratio using Taylor expansion, accounting for the covariance between numerator and denominator.

Ratio metrics A/B test calculator

Question: Is the difference in my ratio metric (like AOV or revenue per click) statistically significant?

Analyze A/B test results for ratio metrics computed as sum(X)/sum(Y) — such as average order value, revenue per user, click-through rate, and more. Uses the delta method for accurate, unbiased variance estimation of ratio metrics.

When to use: For metrics computed as sum(X)/sum(Y) where the denominator varies per user — e.g., AOV (revenue/orders), Revenue per Click, or Time per Session. Users contribute different numbers of events, making simple averaging biased. If your metric is one value per user (like total revenue), use the Continuous Metrics calculator instead.

How to use this calculator

Enter the sample size, mean, and standard deviation for each group. For revenue data, the mean is your average order value or revenue per visitor, and the standard deviation measures how much individual values vary. Most analytics platforms report these values. The calculator uses the delta method to properly estimate variance for ratio metrics, accounting for the correlation between numerator and denominator.

Delta method for ratio metrics

Ratio metrics like AOV (total revenue / total orders) and ARPU (total revenue / total users) cannot be analyzed with a simple t-test on per-user averages because the variance of a ratio is not the same as the variance of individual observations. The delta method provides an unbiased estimate of the variance of a ratio by using a Taylor expansion, accounting for the variances of the numerator and denominator and their covariance. This produces correct confidence intervals and p-values for metrics defined as sum(X)/sum(Y).

When to use this calculator

Use this calculator when your metric of interest is a ratio of two sums — for example, total revenue divided by total users (ARPU), total revenue divided by total orders (AOV), total clicks divided by total impressions (CTR), or any metric defined as sum(X)/sum(Y). This is the correct approach for any metric where each user contributes a numeric value rather than just a yes/no outcome.

Common mistakes with ratio metric testing

The most common mistake is treating ratio metrics as simple per-user averages and applying a standard t-test, which can produce biased variance estimates and incorrect p-values. Other mistakes include ignoring the impact of outliers on highly skewed revenue data, not accounting for the high variance inherent in ratio metrics, and comparing aggregate totals instead of properly defined ratio metrics.