Home Secretary John Reid has ordered a review after a Panorama investigation revealed serious criminals were not being properly monitored after release.
Undercover BBC filming in bail hostels found convicted paedophile and child-killer Frank Parker befriending children while at a unit in Bristol.
Mr Reid has set out plans to improve the probation service, which he feels is underperforming.
He said private sector and voluntary groups can do more routine work.
There are 2,000 offenders living in more than a hundred hostels in England and Wales on court orders and on licence from prison.
Mr Reid told an audience of offenders and professionals at Wormwood Scrubs that: "We've had shortcomings.
"If we don't face up to the fact we've got a problem we are never going to help you."
Too much money is being spent on report-writing and not enough on practical help, the home secretary said.
Mr Reid said he had asked chief inspector of probation Andrew Bridges to review whether the management and operation of the premises mentioned in the programme should be investigated.
"To be frank, the probation system is not working as well as it should," he added.
But shadow home affairs minister Edward Garnier said: "The supervision of post-custody offenders in the community and of serious offenders released on parole has been patchy, to say the least, with terrible consequences in a few cases."
Martin Wargent, chief executive of the Probation Boards Association and a former probation hostel warden, told BBC News the service did a difficult job, with high concentrations of serious offenders in hostels making it hard for staff to provide good levels of supervision.
"Most hostel staff would probably say that's not a good idea," he said.
Secret filming shows Parker befriending children and speaking of taking a photo of a semi-naked teenager who he invited up to his hostel room.
Police tipped off
During filming, Panorama called police twice anonymously to report that Parker was associating with children.
Parker, 61, is not only monitored by the National Probation Service, which runs bail hostels in England and Wales, but also comes under a level of monitoring called Multi Agency Public Protection Arrangements (Mappa).
The police did not pass the details of the calls onto the Probation Service.
When Panorama called 999 a third time and told police staff from the programme had been watching Parker and were concerned about his behaviour, officers rushed to the address given by reporters.
Panorama's investigation focuses on two hostels in Bristol, one of which was home to paedophile Frank Parker, released after serving 39 years in jail for sexually assaulting and murdering a 10-year-old girl.
Since the filming Parker has been recalled to prison.
The National Probation Service says this action was taken not because of what Panorama filmed him doing but because of fears he would abscond after the programme was broadcast.
A second paedophile, Kevin Rogers, who lived at the same hostel as Parker until July - Brigstocke Road - was filmed standing outside public toilets at a Bristol shopping centre and secretly taking pictures nearby.
He is a high risk predatory paedophile convicted of sexually abusing five children and has admitted to abusing others.
Naomi Bryant: Killed by released rapist Anthony Rice
Mary-Anne Leneghan: Four of her killers under supervision
John Monckton: Killer under supervision
Rogers was sentenced to four years in jail but released early to the hostel.
Last summer he moved out of the hostel and into a flat but on 18 July was recalled to prison after he was caught loitering near children at a swimming pool.
Panorama also went undercover at a second bail hostel in Bristol - Ashley House - which, like Brigstocke Road, had a case of an offender killing a member of the public last year.
Ashley House predominantly provides accommodation for people who have committed drug-related crimes. But staff are not allowed to search them for drugs.
Panorama: Exposed: The Bail Hostel Scandal was on BBC One on November 8 2006.