Gordon Brown is under fire over a prison service memo revealing officials have "no interest" in deporting foreign criminals jailed for less than a year.
The UK's head of immigration Lin Homer said staff were committed to sending home all overseas offenders, whether they are dangerous or not.
But the Tories say the memo, sent to prison governors, is at odds with the PM's pledge to expel foreign criminals.
More than 11,000 of the 81,000 prison population are foreign nationals.
The prime minister promised to strengthen measures to deport foreign national criminals in his speech to Labour's annual conference in September.
He told The Times newspaper he wanted a "far tougher line" on foreign offenders.
He said: "I want a message to go out - if you come here to work you learn the language; if you commit a crime you will deported from our country."
But Shadow Justice Secretary Nick Herbert said Mr Brown's words were at odds with the actions of immigration officials.
"It produced a huge headline saying that he was going to remove all of these foreign criminals and that is the impression that the government has repeatedly sought to give.
"And the fact is that we now have this memo that reveals not only are they not removing the majority of these foreign national criminals - but actually that they have no interest in doing so."
Mr Herbert said "foreign thieves, fraudsters, burglars and drugs dealers will be released back into the community" as a result of the policy and he called for all foreign national prisoners to be expelled.
Liberal Democrat home affairs spokesman Chris Huhne, said Mr Brown had been "made to look like a complete chump" by the prison service memo.
"There is no follow through at all. It is a complete failure of joined up government between what the prime minister is saying and what officials are doing," Mr Huhne told BBC News 24.
He added: "It's at odds with common sense. If you have a foreign convict they should be deported at the end of their sentence. It's as simple as that."
UKIP leader Nigel Farage said the "elephant in the room that no one is prepared to talk about" is that prisoners from EU countries will only be considered for deportation after they have served 24 months.
"They have made it illegal for us to deport EU nationals simply because of their criminal record and we cannot even refuse them entry in the first place; even if they are murderers or rapists," said Mr Farage.
Prison Service deputy director general Michael Spurr sent the memo at the centre of the row last month.
In it, he wrote that immigration officials at the Border and Immigration Agency (BIA) had confirmed to the Prison Service that "as a rule they have no interest" in pursuing for deportation foreign nationals jailed for less than 12 months.
The document also says those awaiting removal for immigration offences could be moved to open prisons, to create space in other jails.
Ms Homer, chief executive of the BIA, played down the significance of the memo, saying it was in line with government policy - and offenders jailed for less than a year could still be deported if a court recommends it.
She added: "The prime minister was categorical earlier this year that we are committed to the removal of foreign national prisoners who commit serious crimes in this country.
"Nothing in this document changes that and, in fact, this year we have removed more foreign nationals than ever before.
"We have repeatedly said we will target the most dangerous first, which is why we are initially targeting those who have served more than 12 months."
The Conservatives say that would increase the risk of foreign prisoners absconding, and ignoring prisoners who serve less than a year would mean at least 4,000 criminals would escape deportation.
In October, it was revealed that two prisons, Bullwood Hall in Essex and Canterbury Prison in Kent, had been converted to hold only foreign prisoners.
The Ministry of Justice said the jails, which have immigration and language services, were part of a plan to deport as many foreign prisoners as possible.
In total, 2,784 prisoners from abroad were deported or removed between April 2006 and March 2007.
Ministers first promised tougher measures in 2006 after it was revealed 1,013 foreign prisoners were not considered for deportation last year.
They included murderers and rapists, and the crisis led to the sacking of home secretary Charles Clarke last year.
The UK Borders Act which is due to come into force next year will mean that the type of crime not just the length of sentence will determine whether a foreign criminal will be deported.