Page last updated at 08:50 GMT, Monday, 5 May 2008 09:50 UK

Taking the pulse of the economy

By Ruth Alexander
Reporter, More Or Less, BBC Radio 4

At times of economic volatility, sentiment surveys are headline news.

Female doctor holding clipboard and writing (Picture: EyeWire Inc)
Are sentiment surveys any use as an economic health check?

"Consumer confidence falls to 15-year low", shout the headlines. "Credit crunch puts the squeeze on British business confidence". "Housing gloom 'worst in 30 years'".

While we wait anxiously for official data to be collected, collated, checked, and published, business and consumer opinion surveys fill the gap. How worried are surveyors feeling about the housing market? How do businesses think they're doing? How willing are consumers to spend their money?

And there is no shortage of mood measurers. The Confederation of British Industry, the Institute of Directors, the European Commission, polling companies: the list goes on.

But is it really possible to measure the feel-good factor accurately?

"We're all very impatient to know what's going on in the economy," says Dr James Mitchell, from the National Institute for Economic and Social Research.

"We don't want to be looking in that rear view mirror the whole time. These qualitative surveys are understandably snatched upon by the media and analysts and economists."

Roughly wrong?

Dr Mitchell has been studying sentiment surveys in close detail. What does a more considered look at them show?

"The cost of that timeliness is that, in order to achieve a high turnaround, these surveys only ask people or firms what they think is going to happen in qualitative terms - whether things have got better, stayed the same or got worse."

You might know 80% of company directors surveyed are feeling negative. But not whether they're just a bit downbeat or expecting imminent doom.

Sentiment surveys are relatively easy to run... The media absolutely love them
Ruth Lea, economist

But isn't a rough idea better than no idea? The problem is, some say, a rough idea can lead to the wrong idea.

Economist Ruth Lea used to be head of policy at the Institute of Directors, which publishes a quarterly business sentiment survey.

"You might have a very big balance of members expecting output to go up, but they didn't expect output to go up very much," she says.

"People would interpret that as output going up quicker than if you had a smaller balance of members expecting output to go up. But output could be going up faster in the second case."

You don't know whether companies are predicting output will go up by a mere 1% or a mega 100%.

But how useful are these sentiment surveys for those in the know, who realise they are just broad indicators?

Turning points

The Royal Institution of Chartered Surveyors' March 2008 survey reported that confidence in the UK housing market has fallen to its lowest point in 30 years.

The Rics chief economist, Simon Rubinsohn, says the survey is closely watched.

"The fact that policy-makers, such as the Bank of England and the Treasury, all want to see our surveys in advance of them being published suggests that we are not alone in seeing the benefit of this style of product," he says.

House for sale signs in York, 30 April 2008
The housing market is showing signs of a downturn

"Where it has shown itself to be particularly helpful is in identifying turning points in the market."

Sentiment surveys ask a variety of questions about what has been happening and what respondents predict will happen to house prices, say, or company exports and profits.

Potentially interesting information. But Dr Mitchell says it's best to look at the individual data rather than at a single overall percentage number for general performance or optimism.

If these headline figures have been arrived at by collating and weighting data, as they sometimes are, he says economists can find themselves looking at a signpost pointing in the wrong direction.

The major advantage of these sorts of surveys is that they are quick.

Instead of waiting for the official consumption statistics to come out, for example, why not ask people what they've been buying? That's what consumer confidence surveys do.

Lagging behind

But what quality of information do they get? One person's measure of time can be very different from another's.

This predictive power of consumer surveys is illusionary
Dr James Mitchell, National Institute for Economic and Social Research

"Consumer A might be thinking, 'I've been consuming more these last few months, because I bought a lot last Saturday'," Dr Mitchell says.

"Other consumers might take a very carefully worked out average of their consumption patterns over the last three months and say, 'On balance, I've been consuming a little less this month compared to last month'."

It's hard to know, therefore, that a survey has measured the period it says it's measured.

And because of this, Dr Mitchell says, consumer surveys, in particular, fail to give anything like a forecast, or even a "nowcast".

"This predictive power of consumer surveys is illusionary," he says. "It disappears the moment one evaluates the performance of these surveys relative to the known consumption data themselves."

And Dr Mitchell adds that even when sentiment surveys do get it right, they actually tend to be lagging indicators.

"Many of these consumer and business surveys - published on the understanding that they're telling us about what's happening today, or what might happen tomorrow - are telling us a lot more about what happened to the economy, or to housing prices, six months ago," he says.

"The question then arises, why bother consulting these consumer surveys at all? Why not just look at the historical consumption data or the historical house price data?"

And if you're wondering why anyone does bother with these business and consumer sentiment surveys, hear a confession from someone once on the inside. As a trained statistician, Ruth Lea had strong reservations about these sorts of surveys when she joined the Institute of Directors.

But the attention-grabbing nature of numbers was hard to resist.

"Sentiment surveys are relatively easy to run - especially if you're a membership organisation," she says.

"And certainly in my IOD days, it was very easy to get media coverage when you had a survey. The media absolutely love them. So some of the time we did surveys frankly just to get the coverage."

More or Less will be broadcast on BBC Radio 4 on Monday, 5 May 2008 at 1630. To subscribe to the podcast or listen again, go to


The BBC is not responsible for the content of external internet sites

Has China's housing bubble burst?
How the world's oldest clove tree defied an empire
Why Royal Ballet principal Sergei Polunin quit


Sign in

BBC navigation

Copyright © 2018 BBC. The BBC is not responsible for the content of external sites. Read more.

This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.

Americas Africa Europe Middle East South Asia Asia Pacific