A deal to increase the pay of one million NHS workers may not increase productivity or improve quality of care for patients, experts say.
Nurses were among staff affected by Agenda for Change
A Kings Fund report also warns that Agenda for Change was more costly than expected and implementation was rushed.
New pay bands for all NHS staff, except doctors and managers, were rolled out in 2004 after much negotiation.
The NHS employers organisation said the fairer and more transparent pay system should be celebrated.
The Agenda for Change deal will saw around one million NHS workers' pay increase by around 10% over three years.
As well as rewarding staff more fairly the deal was intended to help deliver fast, modern services for patients.
Researchers at the King's Fund carried out case studies of 10 NHS Trusts as well as interviews with officials, union representatives and managers.
They found that after years of negotiations, national implementation of the new pay structures was rushed.
Furthermore it exceeded all cost estimates with an overspend of £220 million in 2004/5.
And lack of proper implementation mean patients may never feel the intended benefits of the pay reform, they concluded.
However, most managers favour Agenda for Change over the old system and believe it can help them improve productivity and patient care.
King's Fund chief executive Niall Dickson said Agenda for Change was intended to modernise NHS jobs and improve patient care.
"However, this limited but important study shows that as yet there are few signs it has delivered increased productivity or transformed practice.
He criticised the absence of robust, independent audit of the cost and impact of the reform. "Given the scale of the exercise, the millions of pounds of taxpayers money that have been and are being spent, the failure to undertake this is indefensible."
Report author James Buchan said that some trusts were using the reforms to benefit patients.
But added: "Others, dealing with a crowded agenda and nationally led pressure for rapid implementation, have seen it as little more than a 'tick box' exercise, rather than as a way to make services more effectively."
Alastair Henderson, deputy director of NHS Employers, which represents trusts on workforce issues, said Agenda for Change by itself could not bring improvements for patients.
"One of the original intentions of Agenda for Change was to give NHS staff better and fairer pay with a view to improving recruitment and retention.
"This has been achieved and I think it is something to be celebrated.
He added they would be happy to participate in an audit.
Janet Davies, executive director of the Royal College of Nursing, said: "It is still early days and the scale of these reforms means it is going to take some time for all the benefits for patients to be realised - but they will be realised.
"We must remember that at the same time as trying to roll out Agenda for Change, trusts have also been undergoing major structural reforms and some have also been trying to get on top of their financial deficits."
Shadow minister for health, Stephen O'Brien, said: "This damning report exposes the government's failure - a failure which contributed substantially to last year's deficits in our NHS.