Beyond Counting: What Categorical Columns Actually Say

Most categorical analysis stops at a bar chart of counts: region A is biggest, channel B is second. That is a description, not an insight. The interesting question about any categorical column is concentration — how much of everything sits in a few buckets — and it is answerable with two numbers and one classic index.

Start With the Shape

Before any index, look at two things: the number of distinct values, and the share held by the most common value (the top-1 share).

top_1_share = count(most_common) / total_rows

These two together tell you the shape. A column with 200 categories and a top-1 share of 3% is fragmented — no single bucket matters. The same column with a top-1 share of 71% is dominated — one bucket carries the business and everything else is a long tail.

Dominance is not automatically bad: a B2B company may genuinely sell 70% of its volume to one customer segment. But the analysis must treat that fact as the headline, not bury it in a 200-bar chart.

The Herfindahl Index: One Number for Concentration

The Herfindahl–Hirschman index, borrowed from competition economics, is the sum of squared shares:

HHI = sum( p_i^2 )

where p_i is each category's share, expressed as a fraction. Interpretation is intuitive:

• Perfectly fragmented (100 equal buckets): HHI ? 0.01.
• Single category (all rows in one bucket): HHI = 1.
• Two equal buckets: HHI = 0.5. Ten equal buckets: 0.1.

Because shares are squared, HHI responds strongly to the biggest buckets — which is exactly what you want from a concentration measure. A change in HHI between periods is often more informative than the level: it says the mix is concentrating (risk: over-reliance) or diversifying (opportunity).

Putting It to Work

Three places where concentration thinking changes the analysis:

1. Revenue by customer or product. A high-HHI revenue mix means a handful of accounts moving on — or one product failing — moves the whole number. Forecasts and targets should reflect that fragility.
2. Channel mix. A marketing report that shows "channel C grew 40%" is incomplete without the share context. Growing 40% from 2% share is a pilot; growing 40% from 35% share is the business.
3. Long-tail grouping. When a column has hundreds of tiny buckets, group everything below a small share threshold into an "other" bucket before any further analysis. It collapses noise without hiding the shape — and it is exactly what a client-side tool like KPI Master does when it renders composition breakdowns.

Counts answer "what is there". Shares and the Herfindahl index answer "what actually matters" — and that is the question every decision-maker is really asking.