Extreme Comparisons

This is just a fairly short note on what has been covered in other cases elsewhere.  The BBC reported today that Tameside in Greater Manchester is the "UK's heart disease capital".  This is on the basis that the rate of deaths from heart disease between 2009 and 2011 was higher than anywhere else, at 132 per 100,000 people.  The data come from the British Heart Foundation, and I couldn't immediately see how to get at it.

The article mades a particular point of noting that this rate is three times higher than that of Kensington and Chelsea.  This sounds very dramatic, but it really isn't, so don't go moving just yet (even if you can afford a house in Kensington).



The problem with this approach is pretty simple.  If you take two different groups of people, even if the people in these groups are actually essentially interchangeable, different things will happen to them just by chance.  For example, some heavy smokers live to be 90, just because they're lucky (but not many).

If you take a large collection of different groups (all the UK's local authorities, in this case) this effect becomes quite strong.  Let's suppose that people in those groups are essentially the same, say some statistically literate but morally dubious government has decided to randomly assign people to live in particular areas.  Just by chance, some groups will have more cases than others.

I tried this with a little simulation - take 300 large-ish equally sized groups with an average of 150 deaths each; by chance the number of deaths in each group would typically range between 116 and 187, and variations as large as between 100 in the lowest group and 200 in the highest wouldn't be too surprising.  The effect becomes more dramatic if some of the groups are much smaller than others, as is the case with local authorities.

Exactly the same phenomenon is seen here for bowel cancer rates.  It's explained very well with a funnel plot.

Of course the groups of people in different local authorities aren't the same, even on average.  They'll have different age profiles, for example, different lifestyles, income levels, access to healthcare and so on.  Some of these should be controlled for (depending on the point you're trying to make), some perhaps shouldn't.  But comparing extremes as the BBC (egged on by the BHF) does tells us very little indeed.