Chancellor Gordon Brown has pledged to overhaul the tax credit system following stinging criticism in two major reports.
Gordon Brown wants to see the tax credit system improved
Mr Brown told GMTV that he wanted to ensure the IT system "works better" and that tax credit forms were "clearer".
However, the chancellor ruled out the blanket writing-off of tax credit overpayments.
He said that he had "to balance the needs of the country... with the needs of individual families".
But in cases where HM Revenue & Customs (HMRC) was to blame for a tax credit overpayment then debts would be written off, the chancellor added.
Where tax credit overpayment has occurred the chancellor said he wanted to "sort out the difficulties far more quickly than we've been able to do".
The chancellor has found himself under attack over tax credit overpayments.
Tax credits are designed to supplement low income families and are seen as key to achieving the government's pledge to halve the numbers of UK children living in poverty by 2010.
However, critics have argued that the system, with its annual reassessment of awards, is far too complex and open to error.
The Office for National Statistics recently reported that of £13.5bn paid out last year, £1.9bn consisted of overpayments.
On Wednesday, two major reports criticised the administration of the tax credit system.
The Citizens Advice (CAB) report - based on 150,000 cases handled by the charity - said that HMRC had "failed to live up to its own standards of information, clarity and efficiency of service" in the administration of tax credits.
In the most extreme cases, families have been hit with demands to repay thousands of pounds in tax credit overpayment.
Meanwhile, Parliamentary Ombudsman Ann Abrahams reported that she was unable to say how many overpayments were because of government mistakes and how many were down to delays in claimants reporting a change in circumstances.
Ms Abraham's report echoed CAB's call for an amnesty on the recovery of overpayments, particularly those caused by official errors in the first two years of the scheme.
Following the publication of the reports Prime Minister Tony Blair apologised in the House of Commons for any " hardship or distress" caused to families.
Shadow Chancellor George Osborne said: "The reports... paint a devastating picture of the administrative chaos, computer errors and political misjudgements at the heart of the tax credit system."
David Laws, the Liberal Democrat work and pensions spokesman, said that the reports showed that the design of the system was "flawed."