Home Secretary Charles Clarke is due to explain to MPs how 1,023 foreign prisoners came to be released without being considered for deportation.
Mr Clarke has Tony Blair's backing, No 10 says
New figures show 288 of those were freed after the government became aware of the problem last summer.
Mr Clarke said he would not quit over the issue. The Tories said it was not yet clear who was to blame.
The offenders, who were released between February 1999 and last month, included rapists and murderers.
Murderers are released from prison under licence and are meant to be under the supervision of the Probation Service. The rapists should be on the sex offenders register.
Details of the blunder emerged less than a week after Mr Clarke announced new measures to shake up the probation system and put new restrictive orders on violent criminals to help stop them re-offending.
Downing Street said Prime Minister Tony Blair had "full confidence" in both Mr Clarke and Home Office Minister Tony McNulty.
The home secretary revealed on Tuesday that he did not know where most of the foreign offenders were now.
He admitted a growth in foreign prisoners had led to the Home Office taking its "eye off the ball" when the offenders should have been considered for deportation or removal.
The Home Office later revealed that 288 of the 1,023 were released between August 2005 and March - meaning the problem continued after it had been raised with the government.
The National Audit Office told ministers last July that preparations to remove foreign criminals from the UK should begin "much earlier" and not be left until the end of their prison sentences.
A Home Office spokesman said: "Additional resources were directed to this, but the system continued to identify more cases than we could consider.
"Now there are sufficient resources, and we are confident no further convicted foreign nationals will be released in this way."
Mr Clarke said the failure leading to the 1,023 releases was "deeply regrettable" and conceded that people would be angered.
So far the Immigration and Nationality Directorate (IND) has located 107 of the total, leading to 20 deportations.
Among the offenders, three had been convicted of murder, nine of rape, five of sex offences against children, seven had served time for other sex offences, 57 for violent offences and two for manslaughter.
There were also 41 burglars, 20 drug importers, 54 convicted of assault and 27 convicted of indecent assault among those freed.
The Home Office said it did not have full details of offences committed by more than 100 of the criminals, but 237 were failed asylum seekers and 54 were still having their asylum applications considered.
More than 870 were serving at least 12 months and 13 were serving more than 10 years.
7 convicted of other sex offences
57 convicted for violent offences
2 guilty of manslaughter
20 drug importers
54 convicted of assault
27 convicted of indecent assault
Richard Bacon, a Conservative member of the cross-party Commons Public Accounts Committee, asked the questions last October that led to the bungle being uncovered.
He said there had "a dereliction of duty" by the Home Office and the Prison Service to ask whether foreign prisoners, who may not have the right to be in the UK, should be deported.
Pressed by the BBC to explain why he should not resign, Mr Clarke said: "I certainly don't think I have a duty to the public to go - I have a duty to sort this out.
"We don't know exactly where everybody is ... I know where about 100 of those 1,000 now are and we are going through the most urgent cases."
Mr Clarke said the Prison Service and IND had "failed to carry out their responsibilities in the way they ought to have done".
But he said both had taken steps "to lead me to be confident that it is now being done properly".
Shadow home secretary David Davis accused Mr Clarke of initially trying to "smuggle out" the news rather than face MPs' questions, and said it was not good enough to blame officials.
He said he did not think there should be calls for Mr Clarke's resignation as it was not yet clear who was responsible for the error and whether Mr Clarke should have noticed it.
Liberal Democrat leader Sir Menzies Campbell said it was "extraordinary" that so many people convicted of serious offences had "simply disappeared".