A third of all inmates in Welsh prisons have no need to be there, says a senior probation service official.
Swansea has the most overcrowded prison in Wales
South Wales Probation chief executive Ian Lankshear told BBC Wales' Week In Week Out they could be safely managed in the community if better supervised.
Judge Patrick Harrington QC said giving prison sentences to non-violent drug addicts was "utterly pointless".
The Home Office said it had a commitment to providing tough community sentences as an alternative to jail.
Prisons in Cardiff, Usk and Swansea had only 13 free spaces left in August, according to Home Office figures.
The privately-run Parc Prison in Bridgend is already eight over its operational capacity limit. The jail was built for 830 inmates, but now routinely houses more than 1,000.
The prison population in England and Wales is now almost 80,000.
Mr Lankshear told Week In Week Out: "At least a third of the people serving sentences in custody could be safely managed in the community if we had the supervision capacity.
"With the amount of overcrowding that there can be in some establishments, the chances of doing anything meaningful with people - in terms of engaging with the reasons why they commit offences - those chances are reduced, and you end up warehousing people."
The probation service and others in the criminal justice system are particularly concerned about the number of people in prison for non-violent drug offences.
The Prison Reform Trust told the programme that the average cost of housing a prisoner was £37,000 pa, but probation and drugs rehabilitation services argued this money could be better spent.
Mr Harrington, a part-time judge said: "A sentence of four months imprisonment on a drug addict is utterly pointless, it achieves nothing whatsoever - but the judge has his options reduced."
There are almost no prison spaces left in Welsh jails
Mr Harrington said judges were driven to a position where "a custodial sentence becomes the inevitable".
He said more thought needed to go into the alternatives which would rid people of addiction to drugs.
Magistrate Jo Jenkins, who sits on the Home Office's independent monitoring board, said the prison system was so full that many rehabilitation programmes could not be delivered.
"The numbers are going to go up inexorably," she said.
"We are going to be holding people in Swansea police station, for instance, or police cells across south Wales.
"They are certainly not going to receive any better assistance with their re-offending problems there. Unless somebody finds a solution to this then I think society really is going to be the poorer for it."
A Home Office spokesman said: "We will always ensure that there will be prison places for those serious, sexual and prolific offenders who ought to be in prison.
"We are also aware that there are people in prison who ought not to be there, including those with mental health issues and vulnerable women.
"We have outlined our commitment to provide custodial places for those who need to be in prison and to ensure that courts have tough community sentences at their disposal to deal with less serious, non violent offenders."
Week In Week Out's A Waste of Time? is on BBC One Wales on Tuesday at 2230 BST.