Home Secretary David Blunkett has told how he was tempted to celebrate when he heard Harold Shipman had died.
Shipman is thought to have killed at least 215 people
But he quickly realised it would have been inappropriate to "open a bottle" because families of the GP's victims would have felt cheated by his death.
Shipman, Britain's most prolific serial killer, was found hanged in his cell at Wakefield Prison on Tuesday morning.
He was jailed for life in January 2000 for murdering 15 patients while working in Hyde, Greater Manchester.
But Mr Blunkett, in a remarkably frank admission, said his first thought on hearing about the 57-year-old's death was to celebrate.
"You wake up and you receive a phone call - Shipman's topped himself.
"You have just got to think for a minute: is it too early to open a bottle?
"And then you discover that everybody's very upset that he's done it."
Shipman, 57, was given 15 life sentences to run concurrently for the murders, and four years for forging a will.
Dame Janet Smith, who chairs the continuing inquiry into the murders, concluded that she believed Shipman had killed between 215 and 260 people over a 23-year period in Hyde and Todmorden, West Yorkshire.
He was the UK's most prolific convicted serial killer, but always denied his crimes.
After Shipman's death, Jane Ashton-Hibbert, whose grandmother Hilda was unlawfully killed by the GP, said she was angry he had been allowed to die.
She told BBC News: "This seems like an easy way out for him. He never showed any remorse or any guilt and that door is now closed to us."
Mr Blunkett, who made his comments at a lunch for London-based regional journalists, said he had to be "very careful" about what he said in his job.
He paid tribute to his "splendid special adviser" Huw Evans, saying he "restrains my wilder instincts".
A Home Office spokeswoman said: "All we can say is that we confirm the home secretary made those comments."
But Harry Fletcher, assistant general secretary of probation union Napo, said: "It was a very unfortunate remark that will cause offence to many relatives of
the victims who would have hoped that Shipman would one day tell the truth about
"It was a comment best not made."
Mark Oaten, the Liberal Democrats home affairs spokesman, was also critical of Mr Blunkett's remarks.
"Many families of Harold Shipman's victims were devastated to learn of his suicide - now they will never know why he killed their mum or granny or brother or sister," he said.
"Mr Blunkett has nothing to celebrate in the fact that the Prison Service is incapable of detaining people properly for their crimes, and has an appalling suicide record.
"Mr Blunkett should put his cork back in the bottle."