The new Ministry of Justice, which has just taken over some responsibilities from the Home Office, has revealed plans to reduce prison overcrowding.
Lord Falconer will head the Ministry of Justice
Justice Minister David Hanson said sentencing guidelines would be reviewed and suspended sentence orders limited.
Non-dangerous criminals who breach the terms of their release will be jailed for 28 days - rather than automatically serving out the rest of their sentence.
Critics say the new ministry could mean more prisoners are released early.
The ministry has responsibility for prisons, probation and sentencing and looks after 139 prisons, which were holding 80,591 prisoners on its first day of operation.
Earlier this year the prime minister conceded Britain's prisons were "at bursting point".
The Justice Ministry will ask the Sentencing Guidelines Council to review its own guidelines, Mr Hanson said in a statement to the House of Commons.
HOME OFFICE SPLIT
Two ministries instead of one
Home Office to refocus on security, policing, counter-terrorism, immigration, borders and ID cards
Ministry of justice for courts, prisons, sentencing policy, probation
He said that alternatives to prison, such as community sentences, had to be considered when they were "more effective in reducing re-offending and providing payback to the community".
With an additional 8,000 more prison places to be built by 2012, Mr Hanson said: "Our prison building programme will therefore continue to ensure we have capacity to lock the most dangerous prisoners away for as long as they are dangerous and enable sentencers to send people into custody wherever they think this is required."
He also proposed plans to limit the use of suspended sentences to more serious offences, rather than "summary" offences - like stealing a car - which can currently attract a suspended sentence.
A Justice Ministry spokesman said there had been about 7,100 suspended sentence orders in 2005 - 41% of which were from summary offences, despite the fact they had been intended for use with more serious crimes.
Mr Hanson told BBC2's Newsnight programme that the 28-day rule was for "low-level offenders" on community-based sentences, and did not apply to dangerous convicts.
"People who are serious offenders will go to jail for long periods of time, if they're on licence and they break it they will go back to jail for long periods of time," he said.
He added: "This is a positive step to prevent re-offending and to ensure that we have a proper penal policy in place."
Justice Secretary Lord Falconer, in charge of the new ministry, said there was "still considerable room for improvement" in the system.
He added that staff, witnesses and victims "should all feel this is a justice system focused on them and their needs."
While he praised the current justice system as being "admired around the world", he said an "honest" look at it was needed to fix its problems.
But for the Conservatives, Oliver Heald said the new ministry would threaten the independence of the judiciary and could see more prisoners released early from prison.
He said Home Secretary John Reid, who has said he will not serve under Gordon Brown when the chancellor is expected to become the next Labour leader, was "cutting and running" from the split.
And he asked: "What sort of offences will no longer lead to short custodial sentences as you said? What sort of offences will no longer attract suspended sentences?
"Are we really going to be letting off sexual offenders, shoplifters who are drug addicts, people who have committed offences of violence?"
Liberal Democrat justice spokesman Simon Hughes said he welcomed the creation of the new ministry but called for sentences to "mean what they say".
"The two big failures of criminal justice policy in the past have been a prison population higher than any other in western Europe and pious declarations of judicial independence at one moment with ministers criticising judges the next.
"Reducing prison numbers and refraining from criticism of judges would be a welcome start."
The ministry, which incorporates the old Department for Constitutional Affairs, employs 77,000 people and its responsibilities include 595 court houses.
It will have a budget of about £8.8bn for 2007/8, over half of which comes from the existing budget for the National Offender Management Service, as well as existing budgets for legal aid, the court service and other areas.