The penal system is "bloated" and unsustainable, says the report
Changes to the justice system should include reducing prison numbers and closing some jails, while also boosting crime prevention efforts, says a study.
Prison and probation funding could be diverted to tackling the causes of crime, said a report commissioned by the Howard League for Penal Reform.
The report, on England and Wales, said overcrowding threatened to "bring the penal system to its knees".
The Ministry of Justice said crime had fallen by a third in the last decade.
But the prison population has doubled over the last two decades to an all-time high of about 84,000, according to Professor David Wilson, chairman of the commission set up by the Howard League.
"We have got more women, more young people, more people with mental health problems. We have more life-sentence prisoners than the whole of western Europe combined," he told BBC Radio 4's Today programme.
"The costs of locking up those numbers of people are horrendous, and even so when they are released from jail they go on to commit even more crime so we are not getting a good return from our money."
The two-year independent study by the Commission on English Prisons suggested people should consider whether they wanted to live in a "tolerant, pragmatic, forgiving society".
"The alternative is more of the unrestrained and irresponsible penal excess that is storing up an avalanche of future problems for society while spending ever-increasing sums of public money for the privilege of doing so," it said.
It called for the replacement of short-term prison sentences with "community-based responses".
Local prison and probation budgets would be devolved so money could be reinvested in community projects that tackled the causes of crime, it said.
Professor Wilson said: "We could do something far better by diverting some of the billions of pounds that are spent on our criminal justice system, and put that money into the community where we could stop problems happening in the first place."
"The vast majority of the public have lost faith in the criminal justice system, so we have got to reconnect the public, the local community, to having faith in what it is that we can do to create safer communities and less crime."
The Ministry of Justice (MoJ) said: "We do agree that for those who have committed less serious offences, community punishments are highly effective, with a lower re-offending rate than short custodial sentences.
"This is why we have introduced innovative and locally based intensive alternatives to custody which sentencers can have confidence in and which prove effective in turning offenders away from crime."
But a spokeswoman added that prison would always "play a critical role in punishing and reforming and is the right place for the most serious, violent and persistent offenders".
"We have increased the prisons estate and will continue to develop a modern prison service which both punishes and reforms," she said.
The commission reported that lessons could be learned from the "promising" approach in Scotland, which has set itself a goal of reducing the prison population by nearly 40% and has created local Community Justice Authorities.
While the commission said the MoJ should continue to set policies and minimum standards, it proposed setting up a body similar to the National Institute for Clinical Excellence (Nice), which provides national guidance on good health and illness-prevention.
The body for prisons would assess the social consequences of sentencing policies, said the report.
Since the 1990s, England and Wales had been "on a course towards becoming a jurisdiction which punishes excessively harshly and with little attention paid to the relationship between legislation and the impact on prison numbers", Professor Wilson said.
"The result is a crisis of overcrowding which threatens to bring the penal system to its knees.
"Less crime, safer communities and fewer people in prison should be an achievable future."
The report also said more work should be done to make restorative justice work in more cases.
The idea, which brings victims and offenders together, is usually used to deal with lower-level conflicts in schools and neighbourhoods but the commission said if done well it could be applied to more serious offences.
The MoJ spokeswoman insisted its "approach is working", adding that in the last decade crime had fallen by a third, violent crime by further still, and that the risk of being a victim was at its lowest level for a generation.
"People who commit serious offences are going to prison for longer and are being rehabilitated - and the rate of re-offending continues to fall."
Meanwhile, the House of Commons justice committee of MPs has said politics often descends into a "competition as to who can appear toughest on crime", according to sentence length.
But longer sentencing failed long-term by diverting money from crime prevention work, said the committee, which is holding an inquiry into how sentencing policy can be more effective.
A third report, from the centre-right think tank Policy Exchange, has said there is a "postcode lottery" when it comes to youth sentencing.
The report said that the youth jail population could be cut by a quarter, saving £250m, if areas with abnormally high custody rates brought them down.