Stafford hospital was investigated after statistics showed a high death rate
Using patient death rates to judge hospital performance and sort the "good" from the "bad" is unfair and misleading, researchers have warned.
The UK and many other countries use the standardised mortality ratio to rank hospitals, based on how many patient deaths occur above expected levels.
It was this measurement that alerted the authorities to problems at the Mid Staffordshire NHS Trust.
But the Birmingham University team says this measure can be unfair.
To be valid, the measurement must compare like with like.
But the researchers say on BMJ Online that this is not happening and the results are biased.
Although the calculation considers factors that may skew the results - such as a particular hospital serving more seriously ill patients who have a higher risk of mortality - Dr Mohammed Mohammed and his team say these efforts are not enough.
Less than credible
They analysed the methods used by Dr Foster Intelligence, a private-public partnership company which annually publishes standardised mortality ratio (SMR) league tables for English hospitals.
Seven variables routinely checked by Dr Foster in four acute NHS hospitals in the West Midlands were studied.
These were age, gender, deprivation, emergency admission, the presence of other illness, primary diagnosis and the number of previous emergency admissions in the last year.
By their calculations two of the variables - emergency admission and other illness - were unsafe to use because of differences in logging the data and admission practices across hospitals.
And two others - primary diagnosis and the number of previous emergency admissions in the last year - were of dubious value.
Dr Mohammed said: "Any claims that variations in hospital SMRs reflect differences in quality of care are less than credible.
"They should not be used without a massive warning on them."
He said their work urgently needed to be replicated with more hospitals at the national level.
Loss of trust
In an accompanying editorial John Wright from the Bradford Institute for Health Research said: "The attraction of using the hospital SMR is clear. These ratios focus on a clear and important clinical outcome and use routinely available data."
But he said: "These findings undermine the credibility of SMRs and indicate that their role in labelling hospitals as good or bad is unjustified.
"Publicly reported quality measures require accuracy and precision to prevent unfair stigmatisation and loss of trust."
At Stafford Hospital, about 400 more people died between 2005 and 2008 than would be expected.
The trust's initial claim that its method of collecting data was to blame was rejected by the Healthcare Commission.
Instead, it blamed low staffing levels, inadequate nursing, lack of equipment, lack of leadership, poor training and ineffective systems for identifying when things went wrong.
The watchdog said SMRs were a "key indicator of quality in healthcare" and that the case of Stafford Hospital was proof of this.
A Department of Health spokeswoman said: "It is clear that no one single measure can ever capture all facets of the quality of care across an entire hospital.
"However, tracking changes in a hospital through SMRs over time can reveal important information on possible underlying issues in care processes."
Professor Sir Brian Jarman, director of the Dr Foster Unit at Imperial College London, said: "Hospitals that have taken this approach in the US, UK and other countries have gained a useful insight into mortality at their institution and this has been associated with documented falls in mortality."