Prime Minister Tony Blair has said he regretted releasing thousands of prisoners early due to overcrowding, but it was "necessary".
Tony Blair said the extra spaces would alleviate overcrowding
With the prison population breaching the 81,000 mark for the first time, the government said it would release some prisoners 18 days early.
Mr Blair said it was a "very temporary" move as more prison spaces are built.
Tory leader David Cameron accused ministers of an "abject failure" for not heeding warnings on overcrowding.
On Tuesday Justice Secretary Lord Falconer announced that some inmates serving jail terms of four years or less would be released early, due to prison overcrowding in England and Wales.
It could mean 25,000 prisoners or more being released early on licence, although they would not include those convicted of serious sexual or violent offences.
Mr Blair said the decision was made because numbers of prisoners had already reached the "top-end projection" for prison numbers made last year.
This was a "temporary measure while new prison places are being built to ensure we do not breach the regulations in relation to prison conditions".
He told MPs at his penultimate Prime Minister's Questions there were three reasons for the decision:
- a 25% increase in sentencing meant more people were in prison
- there are 5,000 extra people in prison as a result of being recalled because they breached the conditions of release
- there were almost 3,000 people in prison on indeterminate sentences for violent and sexual offences
Mr Blair said: "When we build the new prisons places - a further 8,000 with another 1,500 - we can retrieve this situation.
"I regret having had to do it, but it was necessary to do it."
Mr Cameron said that in 2002 the government was warned that the prison population would be above capacity and called on the prime minister to apologise for "an abject failure to deliver".
"This week we have the scandal of you, in your last few days in office, opening the prison gates and releasing 25,000 prisoners onto our streets," Mr Cameron said.
Last month, Lord Falconer said he would not use early release to ease pressure.
Ministers have been loath to introduce any form of early release, fearing it would dent public confidence in the justice system.
On Monday the prison population reached 81,016, and jails are so full about 400 prisoners are being held in police stations, and about 100 in court cells each night.