President Barack Obama has said Wall Street must repay $117bn (£72bn) to taxpayers and criticised banks for "massive profits and obscene bonuses".
The tax is to recoup money US taxpayers are expected to lose from bailing out the banks during the financial crisis.
"My commitment is to recover every single dime the American people are owed," the president said.
The move follows populist anger at banks, seen as being responsible for causing the recent economic crisis.
"My determination to achieve this goal is only heightened when I see reports of massive profits and obscene bonuses at the very firms who owe their continued existence to the American people," the president said.
He said the aim was not to punish Wall Street firms but to stop abuses and excesses from happening again.
The BBC's Michelle Fleury said the president had made clear "in strong language" that the banks must repay the taxpayer, through what is being dubbed a "financial crisis responsibility fee".
"It may go some way to quelling the anger of the average American," our correspondent said.
The tax would apply only to financial firms with assets of more than $50bn.
There are reckoned to be about 50 of these institutions - although many did not accept any taxpayer assistance and many others have already paid back what the government lent to them.
BBC Business editor Robert Peston said that the cost may be passed on to the banks' customers.
"The cost of borrowing might go up for companies who borrow from these banks," he said.
He said there might also potentially be a "one-off" squeeze on the US economy.
'Drag on sector'
In the US, analysts said the fact that the fees levied on banks would be spread out over a decade would diminish their impact.
"It throws some sand into the gears," said Robert Albertson, chief strategist at Sandler O'Neill in New York.
"It's one more thing dragging on the sector, but it's spread over 10 years, so it's not so consequential. It's petty theft from bank balance sheets."
White House economic adviser Lawrence Summers said the proposed fee was structured to give financial firms incentives to avoid excessive leverage.
The levy comes ahead of the latest reporting season on Wall Street, with banks expected to report record bonuses.
The tax will claw back some of the losses from a $700bn taxpayer bail-out of American banks known as the Troubled Asset Relief Program (Tarp).
It was drawn up in the midst of the financial crisis in 2008, following the collapse of US investment bank Lehman Brothers and multi-billion dollar rescue of insurance giant American International Group (AIG).
It helped stem the crisis by injecting public capital into the biggest US banks and restoring confidence in the banking system.