BBC One controller Jay Hunt told journalists the BBC had "every intention of reimbursing people" who had voted for Sergeant during the series.
Sergeant said dancing on the show was "a wonderfully enjoyable thing", particularly with his Russian partner, Kristina Rihanoff.
"But if the joke wears thin, if in fact people begin to take it very seriously, and if people really are getting so wound up that it's very difficult to carry off the joke, then I think it is time to go.
"It's like when you decide when you leave a party, and the time to leave a party is before the fight starts, and I think that's really what's happened on this occasion," he added.
Rupert Adams, of bookmaker William Hill, told the BBC News website that Sergeant had gone from being a 66/1 outsider to win the series on Saturday morning, to a 10/1 shot by Wednesday morning.
"After Saturday night's programme, people realised that he was a genuine contender," he said.
Sergeant was now 6/1 to change his mind and rejoin the show by Saturday night, he added.
He was put in the most awkward position, looking at the other dancers and knowing they were better than him
The programme's co-host Bruce Forsyth said he felt "very sorry" for Sergeant.
"He was put in the most awkward position, looking at the other dancers and knowing they were better than him. He must have felt guilty in a way.
"It's a bit sad - the whole team adore him," added the entertainer.
The BBC has said it plans to offer to refund the costs of the phone calls to those people who voted for Sergeant in last week's show.
"Because of the unprecedented nature of this we have decided to look into suitable reimbursement," a spokeswoman said, adding it would take a little while to finalise the details.
Strictly judge Arlene Phillips said: "I'm always sad if a contestant leaves of choice, because you are always expecting to let the public vote them in or out - but John is his own person and he has his own reasons for doing this.
John Sergeant explains why he decided to leave Strictly Come Dancing
Phillips pointed out it was not the first time a weak dancer had been backed by the public over the professionals.
In the last series, GMTV presenter Kate Garraway defied the judges, while ex-EastEnders actor Christopher Parker made it to the final in spite of negative feedback.
And she admitted Sergeant had provided many "funny, funny moments".
She added: "Last week's performance was an improved performance. There wasn't a judge who didn't say that."
Chief judge Len Goodman said he thought it was "sad".
"But there was a chance that he could have won the whole competition. And if he did, he did. I'm not one of those that thinks it's terrible, this or that. It's a dance show and if the public like it, well good luck to them," he said.
Meanwhile, actor Chris Parker, who failed to impress the judges in the first series, said Mr Sergeant's impression people didn't want him in the show was false.
"The judges have campaigned for him to be removed, but the public really want him to go through, I think it's a shame.
"People want to see people like John, people who can't dance want to see people who aren't the best. If you had 16 great dancers it would be boring."
Two weeks ago, judges turned on Sergeant, giving his cha-cha-cha, a mere 12 points out of the possible 40.
John Sergeant consistently got the lowest score from the show's judges
Arlene Phillips said the broadcaster was "outstanding at dancing really badly".
Craig Revel Horwood gave Sergeant the lowest possible score - one point - and said his dancing was a "disaster".
However, last week the professionals all agreed the broadcaster's timing had improved after he danced the American Smooth.
Bruno Tonioli said the performance was "surprisingly appealing" and Phillips said he was "quite endearing".
Altogether Sergeant scored 25 - his personal best after the first nine weeks of the series.
Correction 20 November 2008: An earlier version of this story wrongly stated that the money raised from calls to Strictly Come Dancing goes to Children in Need.
Since January 2008, the BBC no longer gives money raised by phone votes to charity. The money raised by Strictly Come Dancing phone votes pays for running the phone vote system itself. No money goes to Children in Need. Rules were changed after last year's phone votes scandals.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.