Mr Osborne said retail banks engaging in risky investment activity would have to set aside "very large amounts of money" as an insurance policy to protect the taxpayer from the cost of a bail-out in the event of a failure.
He said certain "risky" investment banking activities "cannot really easily sit with taking retail deposits", but he stopped short of saying banks that combined the two activities should be split, saying such things were "best done internationally".
"If we just did it in Britain we would see the industry either leave this country or people get round the rules."
According to The Sunday Times, Mr Osborne may float the idea of separate markets regulator, along the lines of the US Securities and Exchange Commission, in addition to the Bank of England's supervisory role.
Mr Osborne has said he believes some banks were allowed to become too big - and he could set out plans to allow the Bank of England to break up banks whose size threatens the stability of the wider economy, the newspaper adds.
We cannot accept a continuation of taxpayers underwriting global investment banking
Vince Cable, Lib Dem Treasury spokesman
On public spending, he told Andrew Marr there had to be a culture change across government - starting with the cabinet - to take account of the fact the "cupboard is bare".
He added: "I have not ruled out tax rises... but I do think after a decade of over-spending, people should not be over-taxed because of that mistake.
"The bulk of the strain in dealing with this debt crisis has to be cutting public spending, restraining public expenditure growth."
The Liberal Democrats are also due to announce their plan for shaking up the banking system on Monday.
Treasury spokesman Vince Cable told The Sunday Times: "We cannot accept a continuation of taxpayers underwriting global investment banking. The only way to stop it is to break up the banks, starting with those that have been semi-nationalised."
Earlier this month, Chancellor Alistair Darling said banks would have to hold more capital and announced plans to strengthen the tripartite system by setting up a new Council for Financial Stability, which would see the FSA, the Bank of England and the Treasury meeting regularly and reporting on the systemic risks to financial stability.
Chief Secretary to the Treasury Liam Byrne said: "David Cameron and George Osborne can talk all they like about banking reform, but when it mattered, they showed their inexperience and called it wrong.
"They opposed the government's action to protect Northern Rock irrespective of the risks to savers and the wider economy.
"Since that misjudgement, they haven't learned their lesson. They're the only politicians in the developed world who think that it's a good idea to cut back spending in the middle of a recession, and deny the economy the boost it needs to get Britain out of the downturn."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.