The Prime Minister has defended the running of the Home Office and the government's record on prisons.
Tony Blair said a process was now in place to deal with "problems"
Tony Blair was speaking earlier in an interview for BBC One's The Politics Show. He conceded prisons were "full to bursting point".
But he said 20,000 new prison places had been created and there would be a further 8,000 added to that total soon.
The prime minister added that prison overcrowding was partly due to people being in jail for longer.
Mr Blair said: "Why are prisons full to the bursting point?
"You've got a situation where, today, people are in prison for longer and you've got, of course, the new indeterminate sentences - where people can be kept in for an indeterminate period if they remain a danger to the public.
"None of us are saying there haven't been big problems at the Home Office but let's be clear, some of these things like foreign prisoners or these offences that have been committed abroad by British people who then return back home.
"The reason we're dealing with these now is that for the first time there is a process in place to deal with them."
In a column for the Sunday Express, Mr Reid had defended his position.
"This isn't a new problem. I am by no means the first home secretary in history to face a burgeoning prison population," he said.
"We have foreign national prisoners, some 1,300 who have finished their sentences, whom I want to deport, but who are occupying much-needed places in the meantime. I make no apology for keeping them in prison in the meantime."
Mr Reid said another factor in the rising prison population was that many more people were being convicted.
"I'm not at all sorry that life can now mean life for some offenders. But it also means that part of the new prisons we build will be occupied by those on public protection sentences."
He said: "Those that want to play yah-boo politics with this issue can just do that... play. Meanwhile, I'll be getting on with the hard work of actually trying to solve the problem."
The Prime Minister's comments come after Lord Chief Justice Lord Phillips - the most senior judge in England and Wales - backed the Home Secretary John Reid over advising judges to consider prison overcrowding when sentencing.
Lord Phillips said Mr Reid was not instructing judges to stop imposing prison terms.
He confirmed that Mr Reid had been simply re-stating existing guidelines, adding that the advice was "helpful".
The advice to judges and magistrates in England and Wales was given jointly by Mr Reid, the lord chancellor and the attorney general.
It prompted two Crown Court judges to release sex offenders, blaming prison overcrowding.
LATEST HOME OFFICE PRESSURES
27 January...The News of the World claims 322 convicted sex offenders are missing across the UK
26 January....Home Secretary John Reid denies telling judges to give softer sentences to ease prison overcrowding
26 January....England and Wales Youth Justice Board head Rod Morgan quits over youth prisons' overcrowding
25 January....Risk of being a victim of crime in England and Wales rises for the first time since 1995, figures suggest
21 January....Proposals reveal the Home Office may be split in two to cover justice and security
14 January Senior civil servant suspended over failure to update police records of Britons convicted abroad
But some judges said they would ignore the guidance when handing down sentences.
Lord Phillips said: "In this statement attention was drawn to the fact that prison population is currently running very close to capacity.
"There is well and long established authority of the Court of Appeal that in such circumstances it is appropriate for the judge to have regard to prison overcrowding."
However, retired judge, Keith Matthewman, told BBC News he had never been asked to consider the prison situation when sentencing offenders.
"I don't know any judge in this country who's ever heard of this being said, that this is something that you take into consideration when you pass sentence," he said.
"You can't take it into consideration. You've got to pass sentence on the crime."
Mr Reid denied that he told judges to give criminals softer sentences to ease prison overcrowding, insisting serious offenders should still be locked up.
A Home Office spokesman said Lord Phillips had reiterated what the home secretary, the lord chancellor and the attorney general have said all along.