Electronically tagging and freeing prisoners as soon as they become eligible could save taxpayers £9.3m every year, say spending watchdogs.
Tagging allows inmates to be freed up to 135 days early
Only six out of every 10 are now freed within two days of eligibility dates, a National Audit Office report reveals.
Prisons, courts and probation should improve the way offenders are assessed for early release, the report says.
It also suggests delays by the private tag-operating firms in reporting curfew breaches increases risks to the public.
Securicor Justice Services and Premier Monitoring Services failed to report nearly one out of every eight breaches to the Home Office within three days, increasing the risk an offender would abscond or commit further crime, the report says.
It took between eight days and two months to return more than one out of every four offenders who breached curfews to prison, the auditors found in a study of 290 cases.
And for one out of every 10 offenders, it took between two months and a year.
The tagging scheme, which allows inmates to be freed up to 135 days early, is a "cost-effective" alternative to jail, the report says.
Tagging an offender for 90 days was £5,200 cheaper than keeping them behind bars, it adds.
Two auditors were tagged for a week as part of their investigation, keeping diaries of their movements to compare with the operating companies' records.
National Audit Office head Sir John Bourn said: "Electronic monitoring represents value for money, providing a cost-effective alternative to custody for offenders who do not pose a risk to the public.
"However, to ensure that a curfew is effective, it is essential that the contractors and criminal justice agencies work together to ensure that offenders are always tagged promptly and that any breaches of their curfew are dealt with quickly."
A Home Office spokeswoman said: "Although performance is generally very good, in some cases there are delays.
"Better communication between the criminal justice agencies involved will do much to reduce these."
A spokesman for Serco, Premier Monitoring's parent company, said: "We are always determined to improve our performance and have done so since the cases sampled by the National Audit Office, which date back as far as 2002 in some cases.
"Currently we are achieving 97% compliance with our contractual targets for fitting tags and 100% compliance for reporting breaches.
"We are pleased that this report recognises electronic monitoring has an important role to play in the criminal justice system and that the technology is robust and effective."
Former Tory minister Jonathan Aitken served seven months of an 18-month sentence for perjury and perverting the course of justice before being tagged.
He told BBC News tagging was effective if properly enforced.
"It does feel as though you are still under surveillance, in a form of custody, because you have to obey the rules.
"You have to be in your home, from six in the evening until six in the morning - not a bad punishment at all for a middle-aged man who likes being at home with his family anyway, but quite a restriction for a young man who likes to go pubbing and clubbing."