Former Conservative leader William Hague has urged Tory MPs not to allow the row over drugs to influence their vote in this week's leadership contest.
Mr Hague said the drugs row should not influence debate
David Cameron has been under pressure to reveal whether he has taken drugs.
Mr Hague said there would be big gaps at the top of most professions if past drug use had been held against them.
Rival Ken Clarke said it was a "silly media witch hunt" while David Davis said it was a distraction from the real policy issues in the contest.
Conservative MPs vote in the first round of their leadership election on Tuesday. In addition to Mr Cameron, Mr Clarke and Mr Davis, Liam Fox is standing.
The candidate getting the fewest votes drops out. A second round of voting on Thursday will then decide which two go through to a run-off vote of the entire Conservative Party membership.
In his column for The News of the World, Mr Hague said: "There are some good reasons for Tories to vote for David Cameron as the next party leader - like the fact that he is obviously able and eloquent. And some reasons not to - like him being inexperienced.
"But even though I favour a much tougher approach to drug laws, I don't think people should be for or against him because he may or may not have taken drugs at university."
Monday: Hustings for all Tory MPs
Tuesday: First round of voting by MPs
Thursday: Final round of voting by MPs
Early November: Tory members start voting on final two candidates
6 December: Result expected
Mr Cameron's three rivals have all said they have never taken drugs.
Mr Cameron has refused to answer drug questions, saying only that he had a "normal university experience."
He also said politicians were "only human".
After newspaper stories regarding a relative of his who had received treatment for heroin addiction, Mr Cameron asked the media to exercise restraint as the situation was not connected to his leadership bid.
Mr Clarke told BBC One's The Politics Show he did not believe the issue was damaging Mr Cameron among the general public, who viewed it as a "silly media witch-hunt".
Mr Clarke said: "It really is quite absurd that any of the candidates were asked to start describing their private lives in former times.
"The moment they answer questions, they immediately get asked about other things, have you done x, have you done y, and all the rest of it. I strongly advise David to carry on batting the questions away."
Mr Davis, on ITV's Jonathan Dimbleby Show, said he believed the Conservative leadership election had been "hijacked" by the issue and that he wanted to concentrate on policy issues.
"This whole thing is a distraction from the real issues - there are real differences between us over policy and so on.
"But this issue is a distraction... it's turning into a hunt and is in my view very much to the disadvantage of the Tory party - not actually to the disadvantage of any particular candidate."
He denied he, or his team, had stirred the issue up.
Meanwhile, George Osborne, Mr Cameron's leadership campaign manager, has denied taking drugs with a woman he is pictured with in the News of the World and the Sunday Mirror.
The 34-year-old MP for Tatton in Cheshire said: "The allegations are completely untrue, and dredging up a photo from when I was 22-years-old is pretty desperate stuff.
"This is merely part of an absurd smear campaign to divert attention from the issues that matter in this leadership contest, and I am confident that people will not be distracted by this rubbish."