[an error occurred while processing this directive]
BBC News
watch One-Minute World News
Last Updated: Tuesday, 30 May 2006, 08:51 GMT 09:51 UK
A glass half empty?

By Lisa Jardine

Statistics are used for everything from advertising shampoos to political arguments over health spending. But how much do we understand about what they mean?

I woke up last Monday morning to a heated argument on the Today programme about the number of NHS hospital beds. In the late 1950s there were a quarter of a million; today there are a little over half that number.

"A good thing" argued the spokesperson for the NHS Confederation: new types of treatment and more effective use of day surgery, have dramatically reduced the length of time a patient needs to stay in hospital. Fewer beds means more money elsewhere in the system and better satisfaction all round.

"Not so", countered the spokesperson from the health-workers' union Unison. Entire wards have been closed under cost-cutting measures. Lobbing statistics back-and-forth between them, both stuck resolutely to their point of view.

How could one set of figures produce two such diametrically opposed arguments? Here was a good example of the fact that statistical data will always allow more than one interpretation. How we view the results will depend on our own attitudes and emotions as much as on the data.

Why are we so easily hoodwinked by numbers? We read our real hopes and fears into claims based on statistics. Advertisers know that an appeal to the aspirations and anxieties of a target audience makes for successful marketing.

In April this year, the manufacturers of Head and Shoulders shampoo were reprimanded by the Advertising Standards Agency for claiming that their product "leaves your hair 100% dandruff free". Dandruff sufferers imagined quite reasonably that if they used Head and Shoulders they would be able to snuggle up, without those tell-tale signs of flaking ruining the embrace.

Hair-raising numbers

But the makers, Procter and Gamble, actually meant was that in trials, Head and Shoulders had reduced flaky scalp in 95 percent of cases, to the point that the dandruff was not visible from a distance of 2 feet away. As for the 5 percent whose scalps had not met even this level of improvement during the timescale of the trials - "a few more weeks' use" would have done the trick.

When is 100% of dandruff really treated?

While this kind of fiddling with the figures will probably be spotted, other, subtler deceptions can be more difficult to recognise. Since antiquity, word-games involving numbers have been used to teach non-logicians to appreciate the pitfalls of arguments based upon them.

The "argument of the heap" - "sorites" - was a favourite teaser of the ancient Greek philosophers. How big is a heap? Ask someone whether a handful of barley - say 50 grains - makes a heap. The answer, of course, is "no".

Now ask: "Well then, if you add a grain to your handful, are 50 grains plus one a heap? No? Then what about 51 plus one?" and so on. You are bound to reach the conclusion that no number of grains of barley, however large, ever becomes big enough to be called a heap.

Incremental changes made to a large population produce no exact point at which we can say, "that is now a heap". Hence the paradox. Words like "many" and "few" lead us by unobjectionable stages to an unacceptable conclusion.

A week or two ago, the British Medical Journal announced the results of a study of the effect of the pattern of women's lives on their health. Newspapers seized on one set of statistics. "Working mums less likely to be obese," the headlines yelled.

Fat chance

"Research suggests that staying at home and giving up work leads to poorer long-term health," explained the Times. "The risk of becoming obese was found to be almost double for a stay-at-home mother."

How clearly were the statistics about Herceptin explained?

I imagine that working mothers like myself smiled smugly and thought, "Well at least we get a bonus from going out to work all our lives - it keeps us thin."

While our home-making sisters sighed, and shook their heads at the thought that ending up overweight was yet another penalty for their decision to devote themselves to the family.

The study had actually found that 38% of women who did not go out to work were likely to become obese by middle age, compared with 23% of women who did.

Of course, no journalist wants a headline that reads: "a few more stay-at-home mothers put on weight, than those juggling jobs and family responsibilities". Because newspaper editors have to grab our attention. And precise-sounding figures apparently do that rather well.

So the advertisement for a well-known after-shampoo conditioner, featuring a gorgeous brunette with shining tresses, proclaims: 'leaves your hair four times smoother!' And the ad for a new brand of mascara solemnly tells us that it makes eyelashes "15 times sexier" - whatever that might mean!

When a new cancer-treatment drug hit the headlines last year, the headline read: "Herceptin halves the chance of breast cancer recurring." It sounded like a miracle. Those like myself recovering from surgery and the rigours of chemotherapy could be forgiven for believing that this meant a 50% reduction in their chance of ever getting breast cancer again, if only they could persuade their NHS trust to pay £20,000 for a course of treatment.

Drug hope

It took me a while to realise that, even though I was one of those suitable for treatment with Herceptin, nobody - not even the surgeon I most trusted and admired - could promise that this would mean the end of all my fears. In fact, once again, the statistics could be interpreted differently - emotionally, felt different - depending on one's circumstances and state of mind.

In the first results from British trials of the new drug, just over 9% of women found their cancer returned. For those who had not been given Herceptin, the recurrence rate was just over 17%. For women like myself, who were temperamentally inclined to believe that their treatments thus far gave them a good chance of total recovery, the new drug seemed to promise a smallish reduction of an already lowish chance.

On the other hand, those women who were most fearful - who felt convinced that their cancer was bound to come back - were sure that Herceptin would slash that likelihood by half. It would give them twice the chance of survival. They believed themselves already to be in that unlucky group who would fall ill again. For them, getting Herceptin on the NHS was, as several of them have said publicly, a matter of life and death.

Working woman, dandruff-sufferer, recovering cancer patient, each of us tries to put the best interpretation we can on the figures, to suit ourselves. Meanwhile, health authorities trying to meet all needs and demands from a fixed pot of money, face a real dilemma.

Should they pay for the cruelly expensive new pharmaceutical for all suitable patients, for the benefit of the most anxious, or are they entitled to argue that the cost per life saved is simply too high?

Drugs like Herceptin can offer hope. But as in the case of the heap argument, there never comes a point at which we are able to say that the greater likelihood of survival is a cure. So who is to arbitrate in these situations - who is to decide if the cost of prescribing it is justified?

We need informed debate - an ongoing dialogue with the medical profession. But if dialogue breaks down, we should not, in my view, take such cases to the courts.

When patients take their health-care provider to court, they are asking the law to take their side, to decide, in cases where the disagreement depends, in the end, not on the statistics, but on strong emotions. Statistics may give us the impression of objectivity, but our decisions based upon them are always, ultimately subjective.

The real problem is inadeqate education so we generally have a population of innumerates. That includes a lot of the media, of medical researchers and particularly of politicians.
Mike McCann, WilmingtonDE USA and Bristol UK

There's lies, damned lies and statistics.... advertisers take take this to the limit so that it becomes lies, damned lies and gutter lies! Deception is the name of the game in marketing. No advertiser is going to say my product is 50% useless... well actually Ratner did about his jewellery and we all know where that led!
aa, Manchester

My favourite recent headline of this sort was "Slave 3 years extra for £3 a week more" used by several newspapers after the recent pensions White Paper, which managed to combine a moral judgement (work being intrinsically bad) with a worst possible - and unlikely - combination of circumstances!
Alastair Scott, London, United Kingdom

Excellent talk. Nice to have something that makes you think - less frequent on R4 than it used to be. Incidentally, the correct answer to the glass half full or half empty question is "the glass is too big". Mike
Mike Sparks, Southampton

It appears that a lot of misconceptions about products come about through the emotive headlines in newspapers. As a newspaper sells itself to some degree by its headlines, it could be argued that the headlines form an advertisement for the newspaper. Surely then the newspapers should abide by ASA standards for advertising, and deliberately misleading people about something like, say, Herceptin results to sell their product, should be punishable by the ASA. Newspapers will, of course, object to this, but the media has, time and again, shown it is incapable of meaningful self-regulation.
Richard, Manchester, UK

When someone quotes statistics to me, I like to say that, if one third of fatal accidents are caused by drink-drivers, then by deduction, it must be twice as danderous to drive when sober!
Bill Erskine, Blandford Forum England

I'd like to echo the comments made about education. Children spend their school mathematics classes learning pointless trigonomentry and simultaneous equations (and I say that having a degree in Mathematics). They should be learning instead how to spot statistical chicanery.
Art, Melbourne

There is a simple rule that can be applied in all of these instances. Percentages unsupported by real numbers are lies. This is proovable time and time again and is the basic way that shonky newspapers and dodgy academics justify their existence. It should be mandated that all claims must be backed by real numbers to enable the viewer/reader to see just how poor the research done by journalists and academics really is. The motto behind all of this should be "put up or shut up"
Chris Waghorn, Wellington New Zealand

As long as you treat statistics like politics or political reporting you will be ok. Always look at what they want to tell you, then why they want to tell you, followed by what they may not be telling you. What you are left with is the usfull stuff, which is in most cases, pointless.
David, Berlin, Germany

My favorite is extrapolation from growth rate. My company had one client in 2004 and two in 2005. That's a 100% growth rate. Did Microsoft have a 100% growth rate in 2005? I think not. Is my company better than Microsoft?
Hélène Dion, Lorraine, Québec, Canada

Thank-you for the article. Jane Jacobs said: "Our downfall will come from Bad Science." It's up to everyone of us to educate ourselves ASAP and to understand figures, randomness, tests, statistics... It's only through education that we can start to comprehend the difficult issues surrounding us. Fear of the dark, witch hunts, and terror are all too common. Let's read, examine, observe, analyse, listen. Only then talk.
Carine, Vancouver

Add your comments on this story, using the form below.

Your e-mail address
Town/city and country
Your comment

The BBC may edit your comments and not all emails will be published. Your comments may be published on any BBC media worldwide.

The BBC is not responsible for the content of external internet sites


Americas Africa Europe Middle East South Asia Asia Pacific