Z-Score vs. IQR: Two Ways to Catch Outliers, and When Each One Fails

Outlier detection sounds like a solved problem until you actually ship it, at which point you discover the two standard methods disagree constantly — and each has a failure mode the other covers. This is why our analytics tooling computes both rather than picking a winner. Here's the mechanics, a worked example where they diverge, and a decision rule for your own data.

The Z-Score Method

The z-score measures how many standard deviations a value sits from the mean: z = (x - mean) / std. The conventional threshold flags anything with |z| > 3. It's fast, interpretable, and grounded in the normal distribution — in truly Gaussian data, |z| > 3 corresponds to roughly 0.13% of observations, so flags are rare and meaningful.

The weakness is structural: the mean and standard deviation are themselves computed from data that includes the outliers. A single extreme value inflates the standard deviation, which shrinks its own z-score. This is called masking, and it's not an edge case — it's the default behavior on heavily contaminated data.

The IQR Method

The interquartile range method ignores means entirely. Sort the data, take the 25th percentile (Q1) and 75th percentile (Q3), compute IQR = Q3 - Q1, and flag anything outside [Q1 - 1.5*IQR, Q3 + 1.5*IQR]. Because quartiles are rank-based, a handful of extreme values barely moves them. The method is robust in the technical sense: its breakdown point is 25%, meaning a quarter of your data can be garbage before the fences fail.

A Worked Example Where They Diverge

Consider ten daily order counts: 20, 22, 21, 23, 22, 21, 20, 24, 22, 95. The 95 is obviously an anomaly — a bulk order, or a logging bug.

Z-score first. The mean of all ten values is 29.0 and the standard deviation is about 22.0 — both dragged upward by the 95 itself. Its z-score is (95 ? 29) / 22 ? 3.0, right at the threshold. A slightly tamer anomaly — say 80 instead of 95 — would score ~2.3 and pass completely undetected. The outlier partially hides itself.

Now IQR. Sorted: 20, 20, 21, 21, 22, 22, 22, 23, 24, 95. Q1 = 21, Q3 = 23, so IQR = 2. The upper fence is 23 + 1.5(2) = 26. The 95 is flagged instantly — along with anything above 26 — and the fence barely notices the outlier because quartiles don't care how extreme it is.

On this dataset, IQR wins decisively. Now flip the scenario.

Where IQR Fails

IQR's robustness becomes a liability on skewed or heavy-tailed data. Daily website traffic with a genuine weekly cycle, or revenue with a long right tail of legitimately large customers, will have a flood of values past the 1.5*IQR fence. Worse, on small samples the quartiles are unstable — with 8 data points, Q3 is essentially one observation, and the fence inherits all its noise. IQR also flags a fixed pattern regardless of sample size; with 10,000 genuinely normal points it will flag far more than 0.13% of them.

Z-score fails differently: it assumes approximate normality and breaks down under masking (above) and under multiple outliers on both tails, which can inflate the standard deviation enough that nothing gets flagged.

For contrast, take 60 days of stable order counts with mean 50 and standard deviation 6 — genuinely clean, roughly normal data. A day at 70 scores z = 3.3 and is correctly flagged, and because the data is uncontaminated, the mean and std are trustworthy witnesses. This is the z-score's home turf: large, well-behaved samples where its thresholds carry precise probabilistic meaning and IQR's fixed fences are comparatively crude.

The Practical Decision Rule

1. Small, possibly dirty datasets (the common case for business exports): trust IQR. Masking is the bigger risk.
2. Large, approximately normal datasets: z-score is well-calibrated and its thresholds have precise meaning.
3. When they disagree, that disagreement is itself information — it usually means the distribution is skewed or contaminated, and you should look at a histogram before trusting either.

That's the reasoning behind offering both side by side: outlier detection isn't a function you call, it's a judgment you make with two independent witnesses. When they agree, move on. When they don't, they've just told you something about your data that neither could say alone.