Charles Clarke is being urged to resign after he admitted the Home Office freed 1,023 foreign prisoners who should have been considered for deportation.
The home secretary insisted he should sort the problem out, and received backing from Prime Minister Tony Blair.
But Tory David Davis and Lib Dem leader Sir Menzies Campbell said Mr Clarke had failed to protect the public.
The prisoners, who were released over the past seven years and include three murderers, are urgently being sought.
Mr Clarke's problems in the Commons were just one element of what was one of the most difficult days for the Labour government since it came to power.
In Bournemouth, Health Secretary Patricia Hewitt was booed and heckled by health workers at the Royal College of Nurses conference.
While Deputy Prime Minister John Prescott was revealed to have had an affair with one of his secretaries.
In the Commons, Mr Davis, the shadow home secretary told MPs he was calling for Mr Clarke's resignation because it had emerged that 288 foreign offenders were released after he knew about the problem in July 2005.
Figures revealed by Sir David Normington, permanent secretary at the Home Office, show that an average of 41 prisoners per month were released between September, 2005 and last month.
Tony Blair is understood to have been told about the releases before Christmas.
Mr Davies told Mr Clarke "that because of the culpable failure to protect the safety of the public, your position is untenable".
Mr Clarke said 83 of the 288 released prisoners were now being considered for deportation, a further 53 had completed the process and 14 had been expelled.
He said he would give details of efforts to track down the serious offenders who were released in error at a later date.
In a statement, he told the Commons the failure of systems within the Home Office was "deeply regrettable" and his priority was now "to set that right".
"It's clear that the increasing numbers of cases being referred for consideration led to the process falling down," he said.
Mr Clarke told MPs he had said sorry for the oversight. "I do apologise. I have apologised, I continue to do so."
FOREIGN PRISONER RELEASES
September - 61
October - 49
November - 34
December - 49
January - 40
February - 30
March - 25
But Mr Davis said although he had known the home secretary for 30 years and had "a broad degree of respect for him", Mr Clarke's statement revealed "a disturbing neglect for public safety at the heart of this government".
"This is yet another example of your department's failure and incompetence. There is no excuse for you not knowing about this," he told Mr Clarke.
And Sir Menzies pressed the prime minister at question time to explain why he had not asked for Mr Clarke to step down.
Tory ex-Home Secretary Michael Howard also urged Mr Clarke to quit after his admission that "it was shocking failure on his part" that had led to the oversight.
"If our traditional principles of ministerial responsibility mean anything at all, he should no longer be in his job," he told the BBC.
Lord Ramsbotham, who was chief inspector of prisons between 1995 and 2001, said he had warned the Home Office that the numbers of foreign criminals in jail were increasing five years ago.
He told BBC Radio 4's PM programme that he drew the government's attention to the problem in his last report in 2001 which was sent to ex-home secretary David Blunkett.
Meanwhile, police have begun processing hundreds of names from the foreign prisoner scandal and have already returned results to the Home Office.
The Association of Chief Police Officers (Acpo) said a team at its London HQ was running Police National Computer (PNC) checks on names being provided by the Home Office.
The checks will reveal if police have current addresses for the missing 900 criminals, and whether any have committed further crimes since their botched releases.
Earlier, Mr Blair said he did not agree with Conservative claims that Mr Clarke was unable to give the Home Office the leadership it needed.
The prime minister has twice refused to accept the home secretary's offers to quit over foreign prisoner blunders.
Number 10 said Mr Clarke had offered to resign on Tuesday when the scandal was uncovered. In interviews, Mr Clarke said he had offered to quit on an earlier occasion as well.
Mr Blair said he regretted the oversights but added that new systems were now in place.
He made the comments after Conservative leader David Cameron claimed Mr Clarke had presided over a "systemic failure" in the Home Office and had misled the public over the scale of the problem.
"Isn't it clear that he cannot give the Home Office the leadership it so badly needs?" Mr Cameron asked, to Tory cheers.
Mr Blair retorted: "It won't surprise you to know I don't agree with that."