The UK's immigration enforcement measures are "clearly inadequate" and need overhauling to take into account a completely changed landscape, say MPs.
Immigration service: Complete overhaul already begun
In a report, they call for employers of illegal immigrants to be targeted in an effort to build public confidence in the system.
The home affairs select committee said an immigration inspectorate with its own Commons committee was needed.
The Home Office said it welcomed the report and shared some of its analysis.
The committee said officials needed to place a "new emphasis" on enforcement controls.
Its extensive report was written before details emerged about plans for the UK's first uniformed border-control force at ports and airports in an attempt to tackle illegal immigration.
It attacked not only the poor quality of decisions in individual cases, but also the apparent inability to remove those who should not be in the UK.
The system regularly failed and a lack of cultural responsibility and clear lines of management meant there was a need for an independent inspectorate, it added.
Home Office removal teams often deported migrants who were settled into the UK with families, rather than targeting the newly arrived, the report said.
However, the committee did not call for an amnesty for illegal migrants, saying it would not work without secure borders.
Committee chairman John Denham said a major test of the government's new approach to immigration would be whether it recognised the reality that illegal migration was being fuelled by people entering the UK legitimately but then overstaying.
Mr Denham said: "Our report highlights many ways in which the administration and decision-making in the system could be improved, but the biggest single change must be a far more focused effort on enforcement.
"A refusal notice should lead inexorably to voluntary or compulsory departure. And the pull factor for new migrants must be reduced by taking effective action against the employers of illegal labour."
Immigration Minister Liam Byrne said the government would this week publish its 20-page report on immigration, a week after announcing that top officials would be sacked in a major review of the Home Office's workings.
15 Home Office directors to be changed and new "top team" set up
Size of Home Office HQ to be cut by 2,700 staff by 2008, with another 600 posts going by 2010
Immigration and Nationality Directorate to become "arms length" agency
Backlog of asylum cases to be cleared in "five years or less"
A key element of the plan is to turn the Immigration and Nationality Directorate (IND) into an arms-length agency with stricter management and targets. Most controversially, ministers have pledged to clear the backlog in asylum cases by 2011.
Mr Byrne said he had carried out an intensive review of how the IND works and spoken to hundreds of staff.
"It's clear to me that not just MPs, but IND itself is frustrated by old ways of working and wants to change dramatically."
Rhian Beynon, of the Joint Council for the Welfare of Immigrants, said the report included some "important home truths" but had failed to accept the case for an illegal workers' amnesty.
"We think exploitative and criminal employers would be more effectively tackled and a culture of workplace rights upheld if irregular migrant workers who suffered exploitation were encouraged to report by being a given a stay against immediate deportation and a grant of temporary leave if their report is vindicated."