The UK Border Agency is looking at thousands of case files
Up to 40,000 immigrants who should have left Britain more than six years ago could still be in Britain, the Home Office has said.
In a letter to MPs, it revealed there was "no formal record" of the people, who were refused permission to stay or whose visas had run out.
The UK Border Agency has begun looking at case files to see if the migrants are still in the UK and can be removed.
These will be checked against police records to see if any are "harmful".
It follows previous rows about the failure to remove foreign prisoners and the backlog of asylum cases.
Agency chief executive Lin Homer said most of the files were from before 2003.
In a letter to the Commons home affairs committee, she said: "In the last few months we have begun the process of reviewing these files to consider if any further action is necessary or possible.
"Where further action is required it will be taken and any cases which may be considered as harmful to the public will be prioritised."
Ms Homer also said: "We believe many of these individuals will either have gone home, been removed following enforcement activity or been granted leave through a different route.
"We expect those that are here illegally to return home. Where they refuse to do so, we will seek to enforce their return."
Shadow home secretary Chris Grayling said: "This looks suspiciously as if the Home Office is trying to get rid of its problem cases by quietly abandoning them in the hope that no one is looking.
"This is a government which has completely mismanaged our immigration system, and now seems to be giving up trying to do anything about the difficulties it has created."
Liberal Democrat home affairs spokesman Chris Huhne said: "Since Labour and the Tories abolished exit checks, it has been impossible to know whether visitors left the country when they should have done.
"It is hardly surprising, therefore, that there are these 40,000 people unaccounted for."
Sir Andrew Green, chairman of Migrationwatch UK, said it was "yet another skeleton in the Home Office cupboard".
He added: "Tens of thousands of case files lying around and the true situation covered up for years on end. This is symptomatic of the utter chaos in the asylum and immigration system during the past 10 years.
"Nobody in the private sector would get away with such a performance."