Sorting out the Home Office "will be difficult" but can be done "over time", Prime Minister Tony Blair has said.
Mr Blair said he had no doubt the problems could be sorted out
There were "tremendous challenges" but people should also recognise the positive changes made since Labour came to power, which had cut crime.
Mr Blair said there would be "radical measures" to improve the Home Office.
Earlier, Home Secretary John Reid said he was not going to quit following a row with judges over sentencing - the latest in a series of controversies.
Last week, Mr Reid was at the centre of a row over sentencing after reminding judges about "existing guidelines" on sentencing, as it emerged that the UK's prisons were, in Mr Blair's words, "full to bursting point".
It prompted two Crown Court judges to decide against jailing sex offenders, blaming prison overcrowding.
On Sunday, it was reported that UK police forces had lost track of the whereabouts of 322 convicted sex offenders.
The story followed others that the Home Office had failed to record the details of thousands of British criminals convicted abroad and also failed to enforce travel bans on 147 drug traffickers.
Mr Blair said: "The Home Office, I've often said, is actually the toughest job in government and that's for a reason."
He added: "What is important to realise is, all the way through, there have been major challenges of benefit.
"That's the reason why crime is down, not up."
But, Mr Blair said: "There are whole set of new issues that we need to deal with.
"It will be difficult. Of course it will be.
"But I have no doubt at all they can be sorted out and will be over time, providing we have got the time for the radical measures to deal with it."
Earlier, Mr Reid told BBC Radio 4's Today programme reforming the Home Office would take two and a half years.
LATEST HOME OFFICE PRESSURES
27 January...The News of the World claims 322 convicted sex offenders are missing across the UK
26 January....Home Secretary John Reid denies telling judges to give softer sentences to ease prison overcrowding
26 January....England and Wales Youth Justice Board head Rod Morgan quits over youth prisons' overcrowding
25 January....Risk of being a victim of crime in England and Wales rises for the first time since 1995, figures suggest
21 January....Proposals reveal the Home Office may be split in two to cover justice and security
14 January Senior civil servant suspended over failure to update police records of Britons convicted abroad
He added: "The fact is I'm in there changing things and I will continue to change them"
"There's one thing that's certain, I'm not going to quit," he added.
Asked directly if he was going to contest the Labour leadership when Mr Blair steps down later this year, he said: "I am going to be home secretary."
Mr Reid said that, under Labour, more serious criminals were being jailed for longer, or given indeterminate sentences.
He said 20,000 new prison places had been created since Labour came to power and said he had commissioned another 8,000 since taking up his job last May.
Writing in the Guardian, Mr Reid likened his Home Office reforms to renovating a house, saying that it is only when you take the wallpaper off "that you discover more problems".
He added: "These problems don't leave me beleaguered. If we weren't discovering more we wouldn't be reforming the Home Office. Indeed I expect more problems."
For the Conservatives, the shadow chancellor George Osborne, said Mr Brown's "Treasury targets, his skewed PFI accounts and his extraordinary decision to freeze the Home Office budget, have all contributed to the current crisis in our prisons and the criminal justice system".
Liberal Democrat home affairs spokesman, Nick Clegg, said: "The British public don't want Bob the Builder as their Home Secretary.
"They want someone who is big enough to admit that the collapse in almost every aspect of our criminal justice system is a direct consequence of 10 years of mismanagement by New Labour."