Justice Secretary Ken Clarke has said it was a "pity" that his attempts to increase the sentencing discount for those who plead guilty was dropped.
Speaking to Evan Davis, he said there were "a lot of silly press stories" around the attempted reform. "The prime minister never ordered me to do anything," he said.
The £100m saving that this reform would have allowed would have to be found elsewhere, he said, but he would not "just go around slashing budgets in all directions".
"The whole cost of the justice system is completely out of hand," he said, outlining how he would be taking £2bn out of a £9bn justice budget, half from administration costs and "quite a lot" from legal aid.
"We're trying to make the criminal justice system work better to protect people. It doesn't protect people because we just let them out of prison and within a year, half of them... are back again."
Among his proposed reforms is the end of the "gimmick" and "unmitigated disaster" of indeterminate sentences.
"I am responding to consultation by repealing them," he said.
Asked about the treatment of Milly Dowler's family, he said he was "extremely sorry" they went thought such an terrible ordeal, but said that it was often an unavoidable consequence of a fair trial.
"You can't stop people making a nasty defence," he said.
Get in touch with Today via
or text us on 84844.