The judge due to hear the treason trial of Uganda's opposition leader has withdrawn from the case, saying there was a perception he would be biased.
Dr Besigye was once President Museveni's personal physician
Judge John Bosco Katutsi described the view as "absolute rubbish" but said a judge should be above suspicion.
He said he had had sleepless nights and his health had suffered.
He is the second judge to withdraw from a case concerning Kizza Besigye - expected to be the strongest challenger in elections due on 23 February.
Dr Besigye faced four different sets of charges, which he said were designed to sideline him in the elections.
The BBC's Will Ross in Uganda says the withdrawal of another judge from the trial of Dr Besigye is likely to raise further public concern over the Ugandan judiciary's independence.
Dr Besigye said he had little hope of receiving a fair trial.
"The whole aspect of attacking the judiciary points to the end result that the judiciary is intimidated. Whoever will judge my case will do so in fear," he said.
A new judge has not yet been named to hear the trial, which is now due to start on 15 February - just eight days before the elections. Earlier this week a constitutional court ruled that Dr Besigye and his co-accused could not be tried on charges of terrorism and weapons offences before a military court martial.
President Museveni remains popular in rural areas
That ruling prompted a Ugandan army general to criticize the judiciary and accuse the judges of siding with "wrongdoers."
Mr Katutsi also heard Dr Besigye's rape trial, which finished earlier this week.
The court assessors recommended that he be cleared of the charges - but Mr Katutsi has not yet ruled on whether or not to he will follow their recommendation.
In November the military deployed while Dr Besigye's co-accused were appearing at the High Court - which the principal judge described as a "defilement of the judiciary".
He was arrested in November shortly after returning from four years in exile.
Dr Besigye, who has been out on bail for the last month, is viewed as the first credible challenger to President Yoweri Museveni, who has been in power for 20 years.
These will be the first multi-party elections since Mr Museveni took power.