BBC News Magazine

Page last updated at 13:43 GMT, Thursday, 30 April 2009 14:43 UK

How one in 1,000 can equal 50%

Sheep Dungeness
Ahhh, lambs gambol in unspoilt rural pastures... wait a minute

Different ways of seeing stats

Michael Blastland

What do you do when that family snapshot of your child playing on a golden beach is ruined by the ugly car park to the right of the picture? Crop it out. Unfortunately, statistical pictures are also heavily cropped to alter the stories they tell, says Michael Blastland in his regular column.

Before I get to the main topic of today's Go Figure column, here's a brief thought about stats and swine flu. There's no statistical debunk of swine flu - at least none that I know of, not yet.

"We don't know" terrifies some people. But many critical numbers are beset by bigger doubts than often admitted. In this case, the one that matters is known as R0 - the case reproduction number, or how many people are infected by each new case - that determines the potential for the spread of the disease.

Sars screening
Sars screening: Containment did make a difference

With Sars, Severe Acute Respiratory Syndrome, which reached near pandemic levels in 2003, this is thought to have been between two and four. That's enough to infect most of the population. In the event, Sars seems to have been controlled sufficiently to stop it reproducing fast enough to spread.

So even if a disease might be dangerous and infectious it is not necessarily true that it will spread once the authorities are onto it.

One problem for those trying to model the spread of disease is that people react to their justifiable fear in un-modelled ways. It's been learnt only relatively recently, for example, that one of the first things frightened people might do is rush to school to collect their children. Those in fear of Sars who sent their children to relatives in the countryside created an effective transmission mechanism.

In short, we simply don't know and the question for the time being is how we react to our ignorance.

• Now, to today's main topic. Frame the picture just right and it shows the house of your dreams. Zoom out and you see the poison factory next door. Framing changes everything. That metaphor captures a sneaky example of rat-like statistical mischief.

Man on bike carrying picture frame
Watch where he puts that frame...

The idea is simple. Change the surrounding view and you can make the number in the middle look vast or tiny. Statisticians sometimes call this "mixed framing", a term I'm going to stretch to apply to a group of sins.

Want to make the output of renewable power appear huge and bountiful? Want to make the output of nuclear power appear small and pointless by comparison? Easy: mix the framing.

Want to make your company's new drug appear powerfully effective? Want to make the fluorescent-rash side effect appear vanishingly rare? Mix the framing.

Here's how it works.

Imagine 1,000 people worried about contracting a smelly disease called Statisticitisis. If they all take a new wonder-pill, their risk is cut by 50%. Sound good?

Meanwhile, one in 1,000 suffer side effects. Trifling, a risk worth taking for such benefit, surely?

Worry ye lots?

Here's the scam. 1 in 1,000 turns out to be the same as 50%. You are as likely to suffer side effects as to be protected from the illness.

How can 1 in 1,000 be the same as 50%? Because only two people out of 1,000 ever have the illness. Cut that by 50% and only one of them has the illness. So one person in 1,000 is protected. But we also know that one person in 1,000 suffers side effects.

One number is a relative risk reduction expressed as a percentage - 50% - and sounds impressive. The other is an absolute number of people - 1 in 1,000 - and sounds like not much to worry about. In fact, they represent the same number of people. That's mixed framing.

Another example from my favourite reading of the moment, David MacKay's Sustainable Energy Without the Hot Air

1. To make an energy source look large, say something like: 'The government's proposed expansion of offshore wind turbines will be enough to power all homes'

2. To make nuclear power look ineffective, say: 'Current plans for nuclear power will cut Britain's emissions by only 4%'

3. In fact, 'power all homes' in the first sentence is the same as the 4% of C02 in the second. Domestic energy use is just part of the UK total energy needs, and produces about 4% of the country's emissions

4. The claims mix the framing (and also switch from a measure of what goes in to what comes out, which really muddies the waters), one using household needs to frame the argument, the other the UK's total C02

Incidentally, MacKay also has embarrassing examples on the other side of the climate change argument

Go Figure's Risk-o-meter (see internet links, right, for previous column) explains how health scares are best turned from percentages into absolute numbers. But this problem goes further. Here, the game is deliberately to swap between one and the other.

It amounts in this case to saying the same thing but with different backgrounds. And it's been shown that in about a third of all drug advertisements and promotions to GPs, drug companies mix framing to make benefits look large and harms look small.

This is a clue for what to watch for with this kind of trickery. If an argument seems to use different language to describe one claim than another, watch out. Percentages versus people in 1000. Households versus UK total. In versus out.

At the simplest end of abuse, a common slipperiness to watch for is in politics, where if spending is intended to look large it will be "£6 billion" and framed against the cost of a single primary school "enough to build…etc"; if small or cheap, it will be "less than half a per cent of GDP to safeguard the interests of the nation."

After a while, it becomes mildly entertaining - though also annoying - to watch for them twisting like this. A slightly nerdy fascination for the units they used last time they had something to say also helps. But - to mix the framing of my metaphors - when numbers are the fundamental language of public argument, it would help if those involved would stick to one language at a time.

Here is a selection of your comments.

Anyone wishing to delve further into this subject would do well to visit the website of Yale academic and graphic artist Edward Tufte, whose book, The Visual Display of Quantitative Information, both describes and illustrates how "mixing the frame" in graphs and graphic art produces spectacularly misleading information.
Barbara Reeve, Canberra, Australia

Some other common wheezes include graphs where the axes are not labelled, so a slight rise of fall in some variable is exaggerated. Another is finding a straight line through a mass of data when the 'trend' is mainly due to one 'outlier', a single data point, or a small cluster of such points. Many of these are to be found in a wonderful book called Facts from Figures by Mike Maroney but I suspect it's out of print now (I couldn't find it in a web search).
Tudor, UK

Whilst at school our maths and chemistry departments were able to prove each year that they both had the best exam results - simply by varying the frame.
Alex Delamain, Faversham UK

Financial organisations would do the same thing if they could to market their products. They cannot because they have to give APR and AER rates which are comparable. Is it feasible to regulate the use of numbers in a particular debate in the same way?
David, Hampshire

And for a particularly topical example, the number of former Gurkhas who would receive the right to apply for residency, compared to the number who, according to a particularly naive projection, might be expected to take up that right...
Ian Kemmish, Biggleswade, UK


The BBC is not responsible for the content of external internet sites

Has China's housing bubble burst?
How the world's oldest clove tree defied an empire
Why Royal Ballet principal Sergei Polunin quit


Americas Africa Europe Middle East South Asia Asia Pacific