Millions of pounds could be saved if government departments tackled 1.4m annual complaints more efficiently, the National Audit Office says.
The NAO is the government spending watchdog
The spending watchdog said significant savings could be made if better mediation techniques were adopted.
Whitehall employs 9,300 people handling complaints and appeals costing £510m a year. £198m goes on legal aid, mainly for asylum and immigration appeals.
The biggest areas for complaints are health, social security and tax.
The NAO found that a complaint cost an average £155 to deal with and appeals £455.
Members of the public see current complaints procedures as "complex, slow-moving, expensive and time-consuming".
Procedures for redress vary widely from department to department.
The report by the NAO - which is entitled Citizen Redress - calls for the Department of Constitutional Affairs to take a leading role in considering methods of improving complaint procedures.
It suggests the department should work with the Cabinet Office and ombudsmen to discover whether there should be a single point of contact to supply the public with impartial information on how to make a complaint.
"Too many members of the public see the way government bodies handle complaints as being complex, slow-moving, expensive and time-consuming," said Auditor General Sir John Bourn.
"Whitehall should convince the public that it is putting their needs and expectations first.
"Better information and greater clarity will not only make it easier for the citizens to get any errors or injustices put right, but should make it less likely that such errors are made in the first place - a better service for the public that brings significant efficiency savings."
The report found that a reduction of 5% of the costs of running complaints procedures would save taxpayers £25m a year.