The Lib Dems said the proposal was "harsh and insensitive"
Politicians from all parties have criticised a government paper proposing to charge interest on emergency state loans given to the UK's poorest people.
The Tories branded the idea to use credit unions "outrageous" and accused ministers of acting like "loan sharks".
The government said the reform was "never our intention", blaming a poorly drafted paper on the social fund, which a minister signed, for the confusion.
But Defence Secretary John Hutton said using credit unions was "a good idea".
The row concerns a Department for Work and Pensions consultation document which proposes changes to the social fund.
It currently extends £500m a year in interest-free loans to some 1.2 million benefit claimants, many of them elderly or disabled.
The document, signed by Work and Pensions Secretary James Purnell, stated that the government was considering offering contracts to credit unions "to take over the provision of credit to social fund customers in their areas".
As well as providing "affordable loans", they could also offer other services such as savings accounts and financial advice, it said.
Work and Pensions Minister Kitty Ussher: 'We are not proposing to charge interest'
"To fund the cost of these extra services, we are proposing that the credit offered under these arrangements could attract an interest charge of 1-2% per month - the same criterion which applies to credit unions."
The paper stated that this monthly rate added up to an annual rate of between 12.68% and 26.8%.
Shadow foreign secretary William Hague told the BBC the proposal was "right out of order".
"This is astonishing. It is outrageous," he said.
"Here we have a proposal under Gordon Brown's government for interest rates up to 27% - the interest rates of the worst store cards or the loan sharks - to be imposed on some of the poorest people in the country."
Defence Secretary John Hutton said it was "ridiculous" to suggest ministers were acting like "loan sharks".
The idea of trying to involve credit unions in this way is quite a good idea
He told Sky News the government was trying to see if there was "a role for credit unions who are not loan sharks".
"The idea of trying to involve the credit unions in this way is a quite a good idea," he said. "The government has no desire to punish people who need these loans."
Since the row broke out on Sunday, the government has tried to distance itself from the DWP document and said that poor drafting had given a false impression people could be charged interest on social fund loans.
Sources have told the BBC: "It will not happen and it was never our intention."
A DWP spokesman said the intention was to consult over widening access to credit unions to people on benefits and those in work to provide advice on debt.
Work and Pensions Minister Kitty Ussher said: "We are not proposing to charge interest on the government loans."
Speaking on the BBC's Andrew Marr Show, Lord Kinnock said there was "no justice" in the plan.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.