Megrahi is the only person convicted over the Lockerbie bombing
The release of the Lockerbie bomber has not given "succour to terrorists", Gordon Brown's spokesman has said.
But he repeated that the decision to release terminally ill Abdelbaset Ali al-Megrahi on compassionate grounds was a matter for the Scottish government.
Meanwhile, Buckingham Palace has confirmed the Duke of York will not now visit Libya on a trade trip next month.
Opposition parties have called Mr Brown's public silence on the issue "cowardly" and "absurd".
Last week Scottish Justice Secretary Kenny MacAskill allowed Megrahi to be released after serving eight years of his life sentence for the 1988 bombing of a US-bound Pan Am flight, in which 270 people were killed.
There was fury from families of the victims and US politicians after Megrahi, who has prostate cancer, returned to his native Libya to huge celebrations.
Justice policy in Scotland is a devolved matter, meaning the Scottish - rather than the UK - government is in charge of it.
However, there has been some criticism that Mr Brown did not intervene to try to persuade Mr MacAskill against Megrahi's release, given the international implications.
Buckingham Palace has confirmed Prince Andrew would not now visit Libya after a proposed trip early next month said to be in the "planning stages" was reviewed by the Foreign Office.
The Duke of York had been set to visit the country in his role as the UK's special representative for international trade and investment on his third official visit to Libya.
Asked about the message that the decision had sent out, Mr Brown's spokesman said: "I don't see how anyone can argue this has has given succour to terrorists."
Pressed over the possible damage to UK-US relations, the spokesman said: "Clearly the prime minister recognises this was a very difficult decision and was clearly an extremely sensitive one, and that there will be very strong feelings from the families."
Challenged over why Mr Brown would not comment, the spokesman repeatedly said that it was a matter for the Scottish justice secretary.
He said: "It would be wrong to reverse that and take a public decision after the decision. It was and remains a decision for the Scottish justice secretary."
He added: "He [the prime minister] found the scenes at Tripoli airport thoroughly distasteful and fully supports what the foreign secretary and Alistair Darling have said, and will continue to work with the Libyans to ensure that those things are not repeated."
Liberal Democrat leader Nick Clegg said: "Although the decision to release Megrahi was a Scottish one for which Gordon Brown was not personally responsible, the fallout puts the UK at the centre of an international storm.
"In these circumstances, it is absurd and damaging that the British prime minister simply remains silent in the hope that someone else will take the flak."
Megrahi received a hero's welcome in Libya
Shadow defence secretary Liam Fox said it was "inexplicable" and "cowardly" that the prime minister had refused to say whether he thought it was right to let Megrahi return to Libya a free man.
"Gordon Brown is willing to give us his opinion on the death of Michael Jackson, he gave us his opinion on the racism row in Big Brother, he has taken time out to give us his views on England's victorious cricket team, but a deafening silence on the release of a mass murderer," he told he BBC.
"A decision which is likely to impact on Britain's reputation for justice and our relations with the United States, our most important partner."
Dr Fox called Mr MacAskill's decision to free Megrahi "wrong," adding: "I think people will view that what is regarded as a compassionate release has been against their sense of natural justice."