It is wrong that judges have become the "whipping boys" over flaws in the sentencing system, Lord Falconer says.
The judge acted in accordance with guidelines in Sweeney's case
The lord chancellor called for a "very urgent" look at automatic discounts given to jail terms for guilty pleas.
Home Secretary John Reid earlier this week denounced the case of a paedophile who could be freed within five years.
Lord Ramsbotham, ex-chief inspector of prisons, urged Tony Blair "to shut up" and halt policy changes which caused more problems than they solved.
It emerged this week that 53 people given life sentences had been freed since 2000 after serving fewer than six years in jail.
Lord Falconer told BBC One's Question Time: "We need to be extremely careful that we don't attack the judges on these issues where it is the system, and it's not one or other political party, it's 30 years of statutes that have led to this.
"We need to be very, very careful to ensure we work together to reach a solution.
"I think the difficulty in relation to it is the whipping boys for this have become the judges and that is completely wrong.
"If we attack the judges, we attack an incredibly important part of the system when it is not their fault."
Referring to the case of paedophile Craig Sweeney - who was jailed for life at Cardiff Crown Court this week but could be released in five years because he pleaded guilty - he added: "Everybody agrees the sentence isn't what we wanted but it wasn't the judge's fault."
'Urgent look needed'
Judge John Griffith Williams QC was acting in accordance with guidelines when he sentenced Sweeney, 24, at Cardiff Crown Court, who had admitted kidnapping and sexually assaulting a three-year-old girl.
Lord Falconer later told BBC Radio 4's Today programme the judge was "compelled" to implement "a series of discounts" which meant Sweeney could be eligible for parole at that stage of his life sentence.
"I think there needs to be a very urgent look at the discount for guilty pleas... we need to look at the system and we need to make changes to ensure this doesn't happen again. We are all agreed about this."
However, Lord Ramsbotham called for the prime minister to stop making changes to the legal system which he claimed created more problems than they solved.
"I just wish he'd shut up, frankly," he told BBC Two's Daily Politics show.
"One of the problems that there has been recently is announcement after announcement from the prime minister that he's going to do this and that and the other, and more people are going to come in [to jail] for longer.
"Unfortunately all that's doing is crowding the system even more that it is, and that doesn't just apply to the prison service.
"It also applies to the probation service, because the probation service is trying to supervise over a quarter of a million people, and they just simply haven't got the numbers or the resources."
Home Secretary John Reid described Sweeney's sentence as "unduly lenient" and on Monday said he would be writing to Attorney General Lord Goldsmith to ask him to consider referring the case to the Court of Appeal.
Asked if the home secretary was right to openly criticise the judge, Lord Falconer replied: "It's quite legitimate for politicians to point out where there is a policy problem.
"John Reid is perfectly entitled to say 'I think the sentence is too low'. He did not attack the judge."
However, Shadow Attorney General Dominic Grieve criticised Mr Reid, calling for him to apologise for his remarks.
Mr Grieve told BBC Radio 4's Today programme "there was no justification" for the home secretary's comments.
"The sentencing exercise that the judge carried out was entirely the result of the government's own 2003 Criminal Justice Act; he followed it to the letter.
"If it produces a result which appears to most people to be very odd and unacceptable that's entirely, I'm afraid, because the government didn't listen."
Lib Dem home affairs spokesman Nick Clegg said public confidence was vital to the operation of the criminal justice system.
He said many sentences currently called "life sentences" were clearly not for life so should be renamed. Only cases where someone was to genuinely spend their entire life in prison should be called life sentences, he said.