Deputy Prime Minister John Prescott has apologised after admitting council tax bills for one of his official homes were paid with taxpayers' money.
Mr Prescott says he thought he was paying the tax
A total of £3,830.52 in public money was used from 1997 to pay tax for the flat at Admiralty House, London.
Mr Prescott, who is in charge of council tax, said it was a mistake and he was reimbursing the government.
The Tories' Caroline Spelman said the public "will find it galling" that Mr Prescott had not paid the bills.
"People could be forgiven for thinking 'there is one rule for Cabinet Ministers and another for the rest of us'," said Ms Spelman, the shadow secretary for local government.
Mr Prescott said the mistake was an "inadvertent error" and that he had thought he was paying council tax on all of his three homes.
Mr Prescott's officials stress he paid council tax on his constituency home in Hull and the Clapham flat which he gave up in 2003.
And he had believed council tax on Admiralty House, near Trafalgar Square, was being paid from the money deducted from his salary as tax on other benefits for his job.
In fact, the government was paying the council tax bills at discounted rates - in line with previous practice, he said.
He explained the mistake in a Commons written answer to Conservative shadow local government secretary Caroline Spelman.
"In 1997, the government carried on paying council tax as before on the flat and claimed a discount (on a second home/company basis)," he said.
"This was not confirmed with me and nor was any advice given to me at the time.
"On reviewing the situation, I am now aware that an inadvertent error has occurred, based on a genuine misunderstanding."
Mr Prescott said he wanted to correct the record and "fully apologise" to MPs. His statement follows newspaper stories about the issue.
He said he was now insisting all future council tax bills were sent to him personally - and telling Westminster City Council Admiralty House should be regarded as his primary home for council tax purposes.
Mr Prescott's spokesman said he did not have to reimburse the public purse for the tax costs but believed it was right to do so as the minister responsible for council tax.
The deputy prime minister was defended by Tony Blair's official spokesman, who said: "It is a classic case of left hand-right hand.
"The matter was drawn to his attention in a public way and he discovered the misunderstanding."
The Conservatives said a "sustained campaign" of parliamentary questions and Freedom of Information Act requests by Ms Spelman had prompted Mr Prescott's admission.
"The public will find it galling that the man who has forced up their bills hasn't had to pay it himself," said Ms Spelman.
Liberal Democrat spokeswoman Sarah Teather said: "It is extremely embarrassing for the government that the man in charge of councils hasn't paid his council tax for nearly eight years.
"We are relieved to hear that Mr Prescott will be paying his bill as soon as possible, but this situation should never have arisen.
"If, as Mr Prescott claims, it was an administrative error, this is simply more proof of the chaotic state of his department."
Exeter pensioner Sylvia Hardy, who was jailed for failing to pay her council tax, said she was "staggered" by the news about Mr Prescott.
"There always seems to be one law for people in government and people in business and another law for people like me," she told ITV News.