Nearly £6bn has been overpaid to tax credit claimants since launch in 2003, HM Revenue & Customs has said.
Tax credit have been dogged by fraud and overpayment
In the 2005/2006 tax year alone £1.7bn was overpaid to tax credit claimants, some of this money is likely to be written-off.
The new tax credit system started in 2003, but it has been plagued by complexity, overpayments and fraud.
But the Revenue says tax credits help millions of low income families back into work and out of poverty.
"Tax credit provide support to 20 million people including 6 million families and 10 million children," said Stephen Timms, chief secretary to the Treasury.
"They have helped to lift 600,000 children out of relative poverty since 1997."
The Revenue added that take-up of tax credit had increased and instances of error fallen.
In total, the Revenue revealed that 1.9 million families were overpaid during 2005/2006, down 56,000 on the previous tax year.
The total amount overpaid fell slightly too, from £1.8bn in 2004/2005 to £1.7bn in 2005/2006.
However, critics of the system say that its is too bureaucratic and in instances of overpayment ends up hurting those who it was set up to help.
Charity Citizens Advice has been highly critical over the scale of overpayment.
"People find themselves struggling to repay large overpayments without receiving proper explanations of how they have arisen, and without being given notice before recovery starts, so they have no time to budget or even challenge the overpayment," Katie Lane, Citizens Advice policy officer, said.
Ms Lane added that the way overpayments are recovered needs to be "reviewed urgently."
There have been reports of widespread problems, with claimants, often on very low incomes, being asked to repay money they had spent in good faith.
Earlier, Liberal Democrat work and pensions spokesman David Laws said that the number of tax credit claimants taking their overpayment case to the Parliamentary Ombudsman has increased tenfold.
Many people have said the fault for their overpayments lay with miscalculations by HMRC staff rather than any mistakes they might have made on their applications.
In addition, the tax credit system has been dogged by fraud.
In late 2005, the online tax credit portal had to close after a sustained attack by fraudsters.