The home secretary has said he will not resign after it emerged 1,023 foreign prisoners had been freed without being considered for deportation.
Charles Clarke said he did not know where most of the people, who include three murderers and nine rapists, were.
He admitted a growth in foreign prisoners had led to the Home Office taking its "eye off the ball".
The Lib Dems have accused ministers of incompetence. The Tories are demanding Mr Clarke answer to Parliament.
Mr Clarke said the 1,023 prisoners, who were released between February 1999 and last month, should have been considered for deportation or removal.
The Home Office later revealed that of those, 288 were released from prison between August 2005 and March - suggesting the problem continued after it had been raised with the government.
The National Audit Office told ministers last July that preparations to remove foreign criminals from the UK should begin "much earlier" and not be left until the end of their prison sentences.
On Tuesday evening a Home Office spokesman said: "Additional resources were directed to this, but the system continued to identify more cases than we could consider.
"Now there are sufficient resources, and we are confident no further convicted foreign nationals will be released in this way."
Mr Clarke had said the failure leading to the 1,023 releases was "deeply regrettable" and conceded that people would be angered by the oversight.
So far the Immigration and Nationality Directorate (IND) has located 107 of the total, leading to 20 deportations.
Among the offenders, five had been convicted of committing sex offences on children, seven had served time for other sex offences, 57 for violent offences and two for manslaughter.
There were also 41 burglars, 20 drug importers, 54 convicted of assault and 27 of indecent assault.
The Home Office said it did not have full details of offences committed by more than 100 of the criminals, but 237 were failed asylum seekers and 54 were still having their asylum applications considered.
More than 870 were serving at least 12 months and 13 were serving more than 10 years.
PRISONERS FACING DEPORTATION:
7 convicted of other sex offences
57 convicted for violent offences
2 guilty of manslaughter
20 drug importers
54 convicted of assault
27 convicted of indecent assault
Richard Bacon, a Conservative member of the cross-party Commons Public Accounts Committee, who asked the questions last October which led to the Home Office bungle being uncovered, said he feared "incompetence" was to blame.
He said there had "a dereliction of duty" by the Home Office and the Prison Service to ask whether foreign prisoners, who may not have the right to be in the UK, should be deported.
Pressed by the BBC to explain why he should not resign, Mr Clarke said: "I certainly don't think I have a duty to the public to go - I have a duty to sort this out.
"It is a massive issue and it's true to say, with the vast growth of foreign national prisoners, we took our eye off the ball.
"The first priority at this moment is to get the situation under control - that is what I'm focusing on.
"We don't know exactly where everybody is ... I know where about 100 of those 1,000 now are and we are going through the most urgent cases."
But Harry Fletcher, assistant general secretary of the National Association of Probation Officers, said the chances of tracking down large numbers of released prisoners was "remote".
Mr Clarke, who is likely to make a statement to the House of Commons on Wednesday, following calls by Speaker Michael Martin, said the Prison Service and IND had "failed to carry out their responsibilities in the way they ought to have done".
But he said both had taken steps "to lead me to be confident that it is now being done properly".
However, Shadow Home Secretary David Davis accused Mr Clarke of trying to "smuggle out" the news rather than face MPs' questions.
"At the end of the day it's not good enough to blame officials, frankly it is a issue which affects the safety of the British public," he said.
He said he did not think there should be calls for Mr Clarke's resignation until it became clear who was responsible for the error and whether Mr Clarke should have noticed it.
Liberal Democrat leader Sir Menzies Campbell said it was "extraordinary" that so many people convicted of serious offences had "simply disappeared".
"All the government's tough talk on crime counts for nothing in the face of this incompetence," he said.
Home Office Minister Tony McNulty said he was "very, very shocked" by the news but he did not think Mr Clarke should resign.
Downing Street says Tony Blair has "full confidence" in both Mr Clarke and Mr McNulty.
"It is unreasonable to expect ministers to know what is going on in every nook and cranny in their department," said Mr Blair's official spokesman.
Former Home Secretary David Blunkett called the problems "astonishing". "My view is that heads should roll," he said.
"There are too many people in the system who simply don't care. I fully support Charles Clarke in getting to the bottom of this."