Barclays in Guernsey was one of many offshore banks targeted by HMRC
HM Revenue and Customs (HMRC) has become more effective at tackling tax evasion in the hidden economy, says the National Audit Office (NAO).
But the NAO's report said more should be done because the chances of being found out were still too low.
It also said that the penalties imposed by the HMRC on tax dodgers were low, with few prosecutions each year.
The NAO recommends that the Revenue continues to target self-employed tradesmen and landlords.
"In the absence of robust estimates of the size of the hidden economy, it is difficult to assess the overall effect of the Department's activities on reducing the amount of tax being lost," the report said.
"Nevertheless, the risk of being detected and the consequent penalties for non-compliance are relatively low, and there are opportunities to tackle the hidden economy more effectively," it added.
HMRC welcomed the recognition that it had become more successful in its efforts to tackle this sort of tax evasion.
"The hidden economy deprives the UK's public services of vital funding and places an additional burden on the majority of honest tax payers who work within the law," said a spokesman.
Overall, HMRC raised an extra £4.40 for every £1 it spent on dealing with unpaid tax in the hidden economy.
This is defined as casual moonlighting, "ghosts" working for cash and paying no tax at all, people claiming welfare benefits fraudulently and businesses not registering for taxes such as VAT.
The NAO report points to the relative success of the recent campaign against taxpayers who have been hiding income in offshore tax accounts.
The "offshore disclosure" campaign led to 45,000 people with offshore bank accounts paying an extra £400m in tax, interest and penalties.
The NAO suggests this approach should be used for other groups of suspected tax evaders.
These include self-employed tradesmen in the building and home repair industries and landlords, the number of whom has grown rapidly to more than one million in the buy-to-let boom of the past decade.
"HM Revenue & Customs has experimented with new ways of encouraging people into the formal economy and it is managing to detect more unpaid tax," said Tim Burr, head of the National Audit Office.
"It could make better use of penalties and secure greater publicity for prosecutions to discourage people from operating in the hidden economy," he added.
One tactic the NAO wants to see used more frequently is straightforward advertising to make people more aware of their legal responsibility to pay tax, and to avoid paying cash in hand to employees who are likely to avoid tax themselves.
It says this approach has already led to an extra 8,000 people registering for tax in the UK and is a tactic that has been very useful in other countries.
Campaigns that were run in 2006 and 2007 were aimed at building tradesmen, hairdressers, car mechanics, taxi drivers, landlords and internet traders.
The emergence of those last two groups as problem areas for the taxman was highlighted by people using the Tax Evasion hotline, which started in 2005.
But despite receiving more than 100,000 calls in its first year, it led to only 2,000 completed investigations in 2006-07.
And this pulled in only £2.6m in extra tax, compared with the original estimate of £32.5m that would be forthcoming.
"A third of the calls received in 2006-07 lacked relevant information and could not be actioned," said the NAO.
"More of the information than expected has been of insufficient quality to help with detecting someone working in the hidden economy," it said.
Despite the widely-advertised fact that the HMRC can impose a penalty of 100% of the unpaid tax, the NAO report reveals that this very rarely happens.
In 2006-07, HMRC raised only £5m, or 3% of the £161m extra tax it discovered, in the form of penalties.
This reflected "both the high proportion of cases (estimated at over half of cases) where no sanction was imposed and the low level of penalties imposed in practice" said the report.
The main reason, said the NAO, was that the amount of tax was small or the tax evader had no money to pay.
"We do use high penalties and sanctions, including those where there are high levels of fraud, but not all hidden economy cases require a high penalty," said HMRC.
There were only 69 prosecutions of "hidden economy" tax evaders in 2006-07, with the cost of the prosecutions outweighing the value of the tax raised from them.
The NAO says more publicity for these cases would improve the deterrent effect.
However the use of "data matching", whereby it compares its own records with other computer databases, has now thrown up the existence of about 300,000 more possible ghosts and moonlighters.
About 500 have been scrutinised in the past two years to see if the data was accurate and a further 20,000 such cases will be looked at this coming year.