Ministers and top civil servants have given a no holds barred account of what they think of each other.
Ministers and civil servants spoke out anonymously
An Institute for Public Policy Research report reveals anger on both sides about what they see as a lack of accountability and qualifications.
There is also a belief by many mandarins that poor performance is not being tackled effectively.
In a rare move, IPPR researchers have interviewed ministers and their current senior civil servants for their views.
The report suggests that while ministers are often not specialists in their fields, their top civil servants are hardly experts either.
And the movement of civil servants from one department to another can prevent a base of knowledge from building up.
Both sides complain that there is a lack of accountability for driving improvement, with failure going unpunished and success rarely rewarded.
The report comes after Home Secretary John Reid described his department as "not fit for purpose".
In their research, the IPPR had unprecedented access to eight ministers and 10 permanent secretaries, who gave their views anonymously.
One minister told the researchers: "I was regularly frustrated by the lack of expertise in the department.
"People complain that we spend too much on outside consultants and others, but often we don't have a choice."
Another minister said: "The most fundamental problem with the civil service is that it is not accountable to anybody.
"It is certainly not accountable to ministers. [The lack of accountability] explains why the pace of change in Whitehall is best described as glacial."
Failure of delivery
One permanent secretary said: "As a group, permanent secretaries have managed to duck accountability.
"Permanent secretaries should be held to account for making sure that their departments are 'fit for purpose', and that they have the right capabilities in place."
Another senior civil servant said: "Internal performance scrutiny is not taken too seriously.
"The 'Performance Partnership Agreements' was effectively a self-evaluation process, with permanent secretaries writing letters to each other and themselves."
But one permanent secretary insisted that officials could be identified and held accountable for their performance above and beyond ministerial responsibility.
"On a case by case approach it is usually fairly easy to see whether a particular failure of delivery was down to the official or down to the political direction," they said.
'No price for failure'
Civil servants also suggest there is frustration with the lack of power held by Cabinet Secretary Gus O'Donnell.
One permanent secretary said: "Because permanent secretaries are accounting officers in their own right, there is no thick line of accountability between the cabinet secretary and permanent secretaries.
"This needs changing. I would like to see the cabinet secretary have the power to remove permanent secretaries."
One minister said: "There is simply no price for failure in Whitehall. No price whatsoever. It is this anomaly that really makes the civil service stand out in comparison to the rest of the public sector."
A senior civil servant added: "The 'accountability fudge' we have now protects ministers and officials.
"Ministers can say 'not me guv', while officials hide behind the minister. This is not in the interest of effective delivery. This is not in the interest of effective government."
IPPR director Nick Pearce said the central problem between the two sides was accountability.
"Ministers are held responsible for everything in their department publicly, but they can't hire and fire civil servants," he told BBC Radio 4's Today programme.
He said ministers could exercise control over their senior civil servants but this would in his view "strengthen an already over-mighty executive".
Another option would be to make clear what ministers and civil servants are responsible for and hold the latter to account.
"The civil service will never achieve consistently high performance without external accountability and effective performance management," added Mr Pearce.
Jonathan Baume, general secretary of the First Division Association, the union for senior civil servants, said the delineation between civil servants and ministers is "quite a fraught one".
"What we need to look at is how the executive, that's ministers and civil servants in the round, are held to account by Parliament," he told Today.
A Cabinet Office spokeswoman said the IPPR report was a largely accurate account of the situation in 2004.
But since then there had been widespread reforms to address many of the issues raised.
Measures introduced by Sir Gus include so-called "traffic light" assessments for departments. The Home Office was condemned for multiple failures in the first batch of reviews published last month.