Page last updated at 01:33 GMT, Wednesday, 18 March 2009

Hospital death data 'misleading'

Stafford Hospital
Stafford hospital was investigated after statistics showed a high death rate

Using patient death rates to judge hospital performance and sort the "good" from the "bad" is unfair and misleading, researchers have warned.

The UK and many other countries use the standardised mortality ratio to rank hospitals, based on how many patient deaths occur above expected levels.

It was this measurement that alerted the authorities to problems at the Mid Staffordshire NHS Trust.

But the Birmingham University team says this measure can be unfair.

To be valid, the measurement must compare like with like.

They should not be used without a massive warning on them
Researcher Dr Mohammed

But the researchers say on BMJ Online that this is not happening and the results are biased.

Although the calculation considers factors that may skew the results - such as a particular hospital serving more seriously ill patients who have a higher risk of mortality - Dr Mohammed Mohammed and his team say these efforts are not enough.

Less than credible

They analysed the methods used by Dr Foster Intelligence, a private-public partnership company which annually publishes standardised mortality ratio (SMR) league tables for English hospitals.

Seven variables routinely checked by Dr Foster in four acute NHS hospitals in the West Midlands were studied.

These were age, gender, deprivation, emergency admission, the presence of other illness, primary diagnosis and the number of previous emergency admissions in the last year.

By their calculations two of the variables - emergency admission and other illness - were unsafe to use because of differences in logging the data and admission practices across hospitals.

And two others - primary diagnosis and the number of previous emergency admissions in the last year - were of dubious value.

Dr Mohammed said: "Any claims that variations in hospital SMRs reflect differences in quality of care are less than credible.

"They should not be used without a massive warning on them."

He said their work urgently needed to be replicated with more hospitals at the national level.

Loss of trust

In an accompanying editorial John Wright from the Bradford Institute for Health Research said: "The attraction of using the hospital SMR is clear. These ratios focus on a clear and important clinical outcome and use routinely available data."

Tracking changes in a hospital through SMRs over time can reveal important information on possible underlying issues in care
A Department of Health spokeswoman

But he said: "These findings undermine the credibility of SMRs and indicate that their role in labelling hospitals as good or bad is unjustified.

"Publicly reported quality measures require accuracy and precision to prevent unfair stigmatisation and loss of trust."

At Stafford Hospital, about 400 more people died between 2005 and 2008 than would be expected.

The trust's initial claim that its method of collecting data was to blame was rejected by the Healthcare Commission.

Instead, it blamed low staffing levels, inadequate nursing, lack of equipment, lack of leadership, poor training and ineffective systems for identifying when things went wrong.

The watchdog said SMRs were a "key indicator of quality in healthcare" and that the case of Stafford Hospital was proof of this.

A Department of Health spokeswoman said: "It is clear that no one single measure can ever capture all facets of the quality of care across an entire hospital.

"However, tracking changes in a hospital through SMRs over time can reveal important information on possible underlying issues in care processes."

Professor Sir Brian Jarman, director of the Dr Foster Unit at Imperial College London, said: "Hospitals that have taken this approach in the US, UK and other countries have gained a useful insight into mortality at their institution and this has been associated with documented falls in mortality."

Print Sponsor

Failing hospital to review cases
17 Mar 09 |  Staffordshire
Failing hospital 'caused deaths'
17 Mar 09 |  Staffordshire
Death rate figures 'wrong'
07 Apr 03 |  Kent

The BBC is not responsible for the content of external internet sites

Has China's housing bubble burst?
How the world's oldest clove tree defied an empire
Why Royal Ballet principal Sergei Polunin quit


Americas Africa Europe Middle East South Asia Asia Pacific