Sending fewer criminals to jail is the key to improving prison and probation services in England and Wales, Home Secretary Charles Clarke has told MPs.
Criminals would have to do unpaid work which was clearly visible to the public, although the most dangerous offenders would be kept in jail.
Prison reform groups backed the five year criminal justice plan but want funding to keep jail populations down.
Tory David Davis said the plans were "minor changes" for "massive problems".
The Lib Dems welcomed the thrust of the package but condemned prison overcrowding as a "scandal".
Mr Clarke told the Commons: "Prison does not work in stopping re-offending."
Unpaid community service work could be a "powerful, effective and tough punishment", he argued.
But the new strategy needed to have public confidence, with offenders rehabilitated but people protected from dangerous criminals.
Double amount of unpaid community work done by offenders
Make community service more visible, with signs next to working offenders
Dangerous criminals still to be locked up
Contracts where prisoners promise not to reoffend in return for housing and jobs when released
Named manager to be appointed for each offender
He plans to double the amount of unpaid work done by offenders to almost 10 million hours.
"Community payback" signs would be placed beside where the offenders worked, and plaques left after the project was completed to make the scheme visible.
Asked about reports that offenders would wear T-shirts with the "community payback" logo, a Home Office spokeswoman said: "We're not going into talking about that."
Mr Clarke said: "Prison is only one way of punishing offenders and it is best for the most serious offenders, particularly those who are dangerous.
"But there are better punishment regimes for others, in particular properly organised community sentences... which offer the best chance of stopping offenders offending again."
Mr Clarke said seriously dangerous criminals would be kept in prison for "as long as necessary".
The government also wants to make prisoners sign contracts promising not to re-offend, in return for help with housing and jobs on release.
Some offenders would also be asked to sign up to rehabilitation contracts to get them back on the straight and narrow.
The plans would also ensure there was a named manager for every offender as part of a national offender management service.
Mr Clarke said the idea of community prisons, to bridge the gap between custody and community, would be developed.
Shadow home secretary Mr Davis criticised the government's record on tackling re-offenders.
He said "community punishment had not proved a success" in the past and he asked how the new strategy would be any different.
"The public at large are not protected from a criminal who receives a non-custodial sentence," he warned.
Lib Dem spokeswoman Lynne Featherstone said Mr Clarke was right to say too many people were being sent to prison.
But she said Labour had since 1997 had time to make inroads into the problem.
She pressed for more housing support for people leaving prison and wanted to make sure rules on disclosing offences did not prevent ex-offenders get work.
Figures compiled last month suggested England and Wales had a higher proportion of the population behind bars than any other western European country.
According to the Howard League for Penal Reform, 140.4 people in every 100,000 of the population are in jail - double the rate of Scandinavian countries.
Last October Home Office figures revealed there were 77,800 inmates - just 373 short of the total capacity.
Mr Clarke said there were "segments" in the numbers in prisons where the numbers were too high, including foreign nationals, those on remand, women, young people and those on short sentences.
Juliet Lyon, from the Prison Reform Trust, said: "By reserving prison for the most violent and serious offenders Charles Clarke will score a double hit: ensuring petty offenders pay back to the community and freeing up prison staff to prevent reoffending and ensure public safety."