More than 30,000 people owe at least £10,000, MPs say
Benefit claimants owe an "immense" £1.85bn in overpayments but the government is recovering less than £300m a year, MPs say.
The public accounts committee is urging the Department for Work and Pensions to improve "significantly" its efforts to recover the money.
More than 30,000 people owe at least £10,000, the MPs' report says.
The Tories called the situation "unforgivable", but the government said debt reclaiming had "improved".
The committee said the DWP had been successful in improving debt recovery procedures, increasing the amount recouped from £180m in 2005/06 to £281m in 2008/09.
But the total debt was continuing to rise - reaching £1.85bn by 31 March last year, an increase of 11% on two years before - because overpayments were still larger than repayments.
The committee warned the situation was likely to get worse as the recession had make people less able to repay.
Income Support claims accounted for more than 70% of all debts. Meanwhile, £9.3m of overpayments of less than £65 were written off during 2007/08 because they were considered too small to justify the cost of recovering them.
The committee's chairman, Conservative MP Edward Leigh, said: "An immense amount of money, currently £1.85bn, is owed to the Department for Work and Pensions by claimants who have been paid too much benefit.
"The size of the debt is increasing, moreover, as the amount of overpaid benefit being clawed back is outstripped by the amount referred for recovery action. The current economic malaise is only likely to make worse the rate at which debt can be recovered.
"If the department is to deal with this rising trend in benefit debt, then it has to improve the way it approaches the prevention of debt. It should also review its procedures for validating claims for Income Support, a benefit which is particularly susceptible to big overpayments."
Mr Leigh said the government should set targets for dealing with those owing large sums of money, which had to be backed up with "more up-to-date methods" for recovering debt."
The report recommended using text messages, e-mails and phone calls to remind claimants they have to inform benefit offices of changes in their circumstances.
For the Conservatives, shadow work and pensions secretary Theresa May said: "Labour need to get a grip. It is unforgivable that while taxpayers are tightening their belts, the government is racking up more debt through poor administration.
"These figures are symptomatic of a benefits system that isn't working. We can't go on like this. We need a change of government to deliver the radical welfare reform this country so desperately needs."
A Department for Work and Pensions spokesman said: "The report recognises that DWP's debt management operations have improved, with recovery increasing from around £180m in 2005/06 to over £280m in 2008/09.
"Additionally 97% of the benefits paid out in 2008/09 were paid out correctly.
"Our new taskforce will address debtors who owe the department over £10,000 and we can take them to court if necessary. However, we accept that there is more we can do and so we will consider the committee's recommendations carefully."