Simpson's Paradox: When Every Segment Improves and the Total Gets Worse

Every segment improved. Mobile conversion rose from 2% to 3%. Desktop conversion rose from 8% to 9%. And overall conversion fell from 5% to 4.5%. Nobody made an error. This is Simpson's paradox: a trend that appears in every subgroup reverses when the groups are combined — because the mix of the groups changed at the same time.

It is not a curiosity for textbooks. It strikes every aggregated rate you report: conversion, churn, click-through, average order value, support resolution time. Anywhere a growing low-rate segment dilutes improving high-rate ones, the headline moves opposite to reality.

The Mechanism in Thirty Seconds

Imagine two traffic sources. Last quarter: 100 desktop visitors converted at 8% (8 orders) and 900 mobile visitors converted at 2% (18 orders). Total: 26 orders from 1,000 visitors — 2.6%.

This quarter both sources improved: desktop to 9%, mobile to 3%. But mobile grew to 4,000 visitors while desktop stayed at 100. New total: 9 + 120 = 129 orders from 4,100 visitors — 3.1%... wait, that improved. Let the mix shift harder: mobile grows to 9,000 visitors at 2.5% (improved from 2%) while desktop's 100 visitors convert at 9% (improved from 8%). Total: 9 + 225 = 234 from 9,100 — 2.57%. Both segments up; headline down. The paradox needs only one ingredient: weight shifting toward the lower-rate group faster than rates improve.

Where It Hides in Practice

• Growth dilutes quality metrics. Expanding into a lower-converting channel or country drags the blended rate down while every existing channel improves. Punishing the team for the blended number punishes growth itself.
• Product mix shifts revenue metrics. Average order value falls when a cheaper product line takes off — even if every product's price and attach rate improved. Same arithmetic, different costume.
• Time aggregation hides it. A year-over-year comparison blends twelve monthly mixes into one. The paradox can appear in the annual number while no single month shows it.

The general lesson matches the correlation-causation warning: an aggregate is a weighted average, and weights move. Before concluding anything from a blended rate, ask what moved — the rates, or the weights?

How to Report Through It

First, always show the segments next to the total. A headline conversion number with no segment table is exactly where the paradox does its damage. Second, when you need one comparable number across periods with different mixes, use a mix-adjusted rate: reweight this period's segment rates by last period's segment shares (or any fixed reference mix). If the mix-adjusted rate rose while the raw rate fell, say so explicitly — "up 0.4 points on a like-for-like mix; the headline fell because mobile's share doubled." Third, track the mix itself as a first-class metric: segment shares over time belong on the dashboard, not in an appendix.

Simpson's paradox is not an argument against aggregates — leadership needs one number. It is an argument against unaccompanied aggregates. Pair every blended rate with its segments and its mix, and the paradox turns from a trap into a routine footnote. Segment breakdowns of exactly this kind are what concentration analysis is for: know where your weight sits before you quote your average.