Places such as the Cayman Islands have a reputation for being tax havens.
Treasury minister Stephen Timms has said many banks continued "to be involved in tax avoidance that goes well beyond reasonable tax planning".
His comments came as the Treasury issued preliminary rules for how banks should handle their tax affairs.
The code would "minimise tax avoidance" and ensure banks had "a clear understanding of the behaviours the tax authorities expect of them," he said.
The guidelines are now going to the banks for consultation.
The rules set out what is expected from banks in the management of their tax affairs, and in their relationship with HM Revenue and Customs.
The code - which will be voluntary - covers both the way that banks operate to minimise tax on their own profits, as well as how they manage the money of their clients.
"As part of the consultation we will be talking directly with banks to develop a shared understanding of the principles that underpin the code and, in particular, what it will mean in practice for banks," said Mr Timms, who is financial secretary to the Treasury.
"This is vital to ensuring that the code plays a part in changing the behaviour of banks and in turn minimising the loss to taxpayers through tax avoidance."
He added that banks played a vital role in the UK and were important contributors of tax.
WHAT IS A TAX HAVEN?
Low or no taxation
Lack of transparency
Refusal to provide information to foreign tax authorities
Earlier this year the TUC called on the government to force High Street banks to come clean about their activities in tax havens.
It said its research showed that the four biggest banking groups had more than 1,000 subsidiary companies in tax havens dotted across the world.
Pressure has mounted on banks, especially those which have received government bail-outs, to stop avoiding UK tax, and enabling their wealthy clients to do likewise.
"The government believes that, in the light of the significant taxpayer support provided to stabilise the banking system, taxpayers are entitled to expect that banks, important taxpayers in their own right, and their customers pay their fair share of tax," the code said.