The numbers of foreign nationals in British prisons have increased by 75% over the past five years, Home Secretary Charles Clarke has told MPs.
Exeter Prison holds 537 people but is designed for 316
More than 9,000 of the 77,774 inmates in prisons across England and Wales were foreign nationals, he said, adding that he found the numbers "surprising".
He said freeing prisoners early on an electronic tag may be "undesirable but necessary" to ease prison overcrowding.
For the Tories David Davis called for an urgent plan to resolve "the crisis".
Home Office ministers are already looking at the possibility of freeing non-violent prisoners on an electronic tag six months early, compared with the present four-and-a-half months.
The number of prisoners in England and Wales reached an all-time high of 77,774 last Friday - up from 73,200 at the beginning of 2005.
Mr Clarke, giving evidence to the Commson home affairs committee, said the figure was "very close" to reaching the absolute maximum in England and Wales of just over 78,000.
"It would be undesirable in principle to change the home detention curfew arrangements but that also may be necessary," he told MPs.
"Twelve to 13% of the occupants of British prisons at the moment are foreign nationals - that is a very large number indeed," he said.
"From 2000 to 2005, the number of British nationals in British prisons increased by 11% while the number of foreign nationals increased by 75% over that period.
"If the foreign nationals had increased at the same rate as the British we would have about 3,500 less prisoners than we do today.
"Or to take another example - there are about 800 British prisoners in other EU prisons and about 1,800 EU nationals in British prisons.
"I'm not sure why there is that disparity and we are looking at it."
Shadow home secretary David Davis said: "Charles Clarke has now admitted that the huge rise in immigration under Labour has contributed to the overcrowding of Britain's prisons.
"The Government's policy of encouraging immigration has implications and this is one of them."
He added: "Despite the fact that the Conservative Party has long warned the Government that they have built too few prison places, they have arrogantly ignored the calls.
"Now prisoners who should be punished through a jail term will not be. The home secretary must set out an urgent plan outlining what he is going to do to resolve this crisis."
During the hearing Mr Clarke was also asked whether the government's plans to build accommodation centres for asylum seekers had been completely scrapped.
"They haven't been entirely scrapped because there's always a possibility that they may become necessary," he said.
Documents presented to the committee show that the Home Office's Immigration and Nationality Directorate reported a £19.5m underspend in its budget for 2004-5, due to the cancellation of the accommodation centre planned for Bicester in Oxfordshire.
Ministers have spent £18m on the project since 2001.
Quizzed about the threat of terrorism to the UK, the home secretary added: "For some considerable time we will be in a position where there are people seeking to launch terrorist attacks on this country."