Public sector pay has proved contentious
David Cameron says he does not back a freeze on all public sector pay - but says it cannot be "independent" of pay restraint in the private sector.
The Tory leader told the BBC it was "right" to leave pay levels to the independent pay review bodies but they would be "much tighter" in future.
Audit Commission boss Steve Bundred has said all public sector pay should be frozen to rebalance public finances.
TUC chief Brendan Barber said an across the board freeze was not realistic.
On Sunday Mr Bundred told the Observer politicians had to admit that "severe pay restraint" was one of many measures needed to rebalance the public finances - particularly if pay in the private sector was falling.
"At a time when inflation is likely to be between 2% and 3%, a pain-free way of cutting public spending would be to freeze public-sector pay, or at least impose severe pay restraint," he said.
Asked whether he supported a freeze, Mr Cameron said: "I don't think that is the way we do pay in this country.
"The way we do pay in this country in the public sector, rightly, is we have independent pay review bodies."
Those bodies already took account of pay in the private sector - because they had to consider how to recruit and retain public sector staff, he said.
"Everybody knows public sector pay is going to have a much tighter than it has in the past. Here we are in 2009, we don't know exactly what inflation will be in the future so I don't think it's the right time to make this decision."
In his article Mr Bundred said public sector workers had "done well" in the past 10 years and would "tolerate" modest reductions.
But Brendan Barber, general secretary of the TUC, told the BBC millions of public sector workers had seen "real cuts" in pay over the last three years.
"It's not that long ago that inflation was 5% and the government had a pay policy of 2%, so the prospect of more suffering to come is not an attractive one," he said.
He said it was not "realistic" to suggest an across-the-board freeze as public sector workers ranged from very low paid employees to high court judges.
He said there seemed to be a mindset that there should be an "equality of misery" between the public and private sectors. He called this "absolutely wrong" and said it was "astonishing" Mr Bundred had suggested a freeze would be a "pain free option".
"The prospect of a real living standards being cut for public service workers will obviously provoke a very strong reaction."
On Sunday Chancellor Alistair Darling said pay policy would be decided over the next few weeks adding: "Public sector pay obviously has got to reflect prevailing conditions, and in particular inflation has come way down."
He added: "It has got to be fair to people who work for the public sector just as we have to be fair to the private sector."
Gordon Brown faced union anger in 2007 when he said public sector pay rises had to be held down to prevent inflation, as did a 2% cap on pay rises last year.
Liberal Democrat Treasury spokesman Vince Cable has said an "absolute across-the-board freeze" in public sector pay did not make a "great deal of sense" because of different contractual obligations but bonuses should be stopped.