Beyond Deleting Outliers: Winsorization and Honest Treatment

The analysis looked great after removing "a few bad data points." Which points? "The ones that were clearly wrong." This conversation happens constantly, and it is the easiest way to lie with statistics while feeling rigorous — because sometimes the points are wrong (a sensor glitch, a test order), and sometimes they are the most important observations you have (the whale customer, the outage). Detection is only half the job; treatment is where integrity lives. Winsorization is the workhorse honest treatment: cap extremes instead of erasing them.

What Winsorization Does

To winsorize at the 5th/95th percentiles: every value below the 5th percentile becomes the 5th-percentile value; every value above the 95th becomes the 95th. The extremes keep their direction (high stays high) but lose their leverage — no single observation can drag the mean arbitrarily far. Unlike deletion, the sample size stays intact, so standard errors and tests remain valid-ish; unlike ignoring outliers, one glitch cannot dominate the headline.

Compare the alternatives on one axis — what happens to the extreme observations:

• Deletion: removed. Simple, dangerous: valid extremes vanish, sample shrinks, and the criterion ("clearly wrong") is rarely reproducible. Reserve for proven errors with a documented cause.
• Winsorization: capped at percentiles. Keeps direction and sample size; the capping points are data-driven and reportable ("winsorized at 1%/99%").
• Trimming: removed symmetrically (top/bottom x%) before averaging. Honest when the trim share is fixed in advance — same spirit as the trimmed mean.
• Robust estimators: median, MAD, Huber means. Sidestep the treatment question by using statistics outliers cannot move much — often the cleanest answer when you only need a summary.

Choosing the Caps

Common choices are 1%/99% (light touch, kills only true monsters) and 5%/95% (noticeable stabilization). The choice should follow the data's story, not the desired result: set caps from domain knowledge (latencies above the timeout are impossible — cap there), then disclose them next to the number. Capping at percentiles of this sample is fine for description; for monitoring and alerting, freeze the caps from a reference period, or every new extreme quietly redefines "extreme."

Always run the analysis both ways and report the gap: "mean $71 raw, $58 winsorized at 5%/95%." If the conclusion survives both, it is robust; if it depends on the treatment, the treatment is the finding and must be disclosed, not buried. This pairs naturally with a proper detection pass first — winsorize the points a principled detector flagged, not the points that spoil the narrative.

What Never to Do

Three treatments are never honest: deleting points because the result "looks better" without them; capping asymmetrically (winsorizing the inconvenient tail only); and silent treatment — any capping, trimming, or deletion undisclosed in the methodology note. Outliers often carry the signal (anomalies are how fraud, outages, and breakouts first appear). The goal of treatment is never to make data well-behaved — it is to keep one tail observation from impersonating the whole distribution while leaving the evidence on the record.