David Cameron has said he will not sack shadow cabinet member Alan Duncan for saying that MPs are living "on rations" following reforms to their expenses.
Mr Duncan, the shadow Commons leader, was secretly filmed complaining about the changes and has since offered an "unreserved" apology.
Mr Cameron said: "Alan made a bad mistake and he acknowledges that."
But he added that MPs had to "demonstrate completely" that they understood public anger over expenses.
The secret film, made by Heydon Prowse of Don't Panic magazine, shows Mr Duncan using strong language .
Discussing reforms to expenses, he says: "No-one who's done anything in the outside world or is capable of doing such a thing will ever come into this place ever again the way we're going."
He apologised after the film was widely shown on the internet and television.
Mr Cameron said he would not sack Mr Duncan, one of the best-known members of his shadow cabinet.
Film maker's secret recording
Speaking in his constituency of Witney, Oxfordshire, he added: "I spoke to Alan Duncan yesterday and made it clear in no uncertain terms that when it comes to the mess of expenses, the words we use, just as the actions we take, have got to demonstrate completely that we share the public's real fury at what went on in Parliament.
"Alan made a bad mistake and he has acknowledged that, he has apologised and withdrawn the remarks."
Earlier, the former parliamentary sleaze watchdog Sir Alistair Graham said Mr Duncan's comments could be offensive to people struggling in the recession.
He added that it was "particularly worrying" from a potential minister.
Labour MP Sir Stuart Bell, who sits on the Commons Estimate Committee, which deals with parliamentary pay and expenses, said: "Not only has Alan Duncan been a roadblock to reform, he has now shown how arrogant and out of touch he is.
"How can David Cameron keep him in a job so closely involved with MPs' pay and allowances?"
But Mr Duncan's fellow Conservative MP Sir Patrick Cormack defended him, telling BBC Radio 4's The World at One: "We don't want a parliament of political anoraks and rich people.
"Those who have a vocation for public life shouldn't be deterred from going into parliament because they can't afford to contemplate it.
"Being an MP is an extremely expensive business."
Mr Prowse was invited to the Commons by Mr Duncan after he was filmed digging a pound-shaped hole in the Rutland and Melton MP's lawn in protest at his expenses claims.
The campaigner said he had taken the secret camera along to the meeting in July in the hope of catching Mr Duncan out but he regretted getting him into trouble because he was a "very charming and generous guy".
But Mr Prowse said he thought it was important to expose the attitude of politicians towards the expenses scandal.
As shadow Commons leader, Mr Duncan has overseen much of the Conservative response to the expenses row and has said he will pay back more than £4,000 of allowances.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.