The Lord Chief Justice has called for the restoration of public confidence in community sentences to end the "warehousing" of criminals in prisons.
Lord Woolf believes prison should be reserved for most serious cases
This could be done by ensuring such punishments are policed properly, Lord Woolf told a Home Affairs Committee hearing into sentencing guidelines.
The over-use of custodial sentences had "undoubtedly adversely affected the criminal justice system", he said.
Prison should not be used for less serious, non-violent crimes, he added.
"What has happened is the resources which could have been used in having really effective community punishments have been deployed in warehousing an ever-increasing prison population," England and Wales' most senior law lord told the committee.
There was a vicious circle created by the over-use of custodial terms, Lord Woolf said.
The UK's prison population stands at its highest in history at around 75,000.
The over-use of prison terms meant the ability to properly police community sentences was reduced because resources were being used up on "expensive" custodial sentences, he said.
"If a community punishment is given and nobody follows it up then what's given isn't going to work," the law lord said.
The Sentencing Guideline Council and a government-appointed advisory panel will spend the next five years drawing up a code for judges to use when they hand down punishments in court.
Lord Woolf said he hoped the new procedures would help avoid politicians and judges responding to "public clamour" in particularly horrific cases.
But he urged parliamentarians to exercise caution in rushing to change the guidelines through legislation once they were published.
"There have been instances where legislation made in haste has not been the most constructive way of dealing with situation"
It was also essential that judges retained an element of freedom to take into account the circumstances of individual cases, he added.