Former foreign secretary Robin Cook has cast doubt on Clare Short's claims that the UK spied on UN chief Kofi Annan.
Several former ministers have criticised Clare Short
He said he "would be surprised" if it were true Britain had intercepted the calls of Kofi Annan and he urged Ms Short to "put up or shut up".
Mr Cook is among a group of former ministers who have rounded on Ms Short for her outburst which they say is part of efforts to undermine Tony Blair.
But the ex-international development secretary said she stood by her claims.
She denied putting the UK or its security services at risk by saying Mr Annan's phone calls were bugged.
And after Tony Blair branded the claims "deeply irresponsible", Ms Short hit back by accusing the prime minister of using "pompous" distraction tactics.
She told BBC Two's Newsnight there was no national interest that justified spying on the UN secretary general.
Meanwhile Richard Butler, the former UN chief weapons inspector, claimed at least four countries bugged his conversations and he is convinced the UN's headquarters in New York is full of spies.
UK Home Secretary David Blunkett said he had never been shown transcripts of Mr Annan's phone conversations, despite being "one of the very few people ... who have clearance for the full security
material that comes through".
Pressed about whether Ms Short had breached the Official Secrets Act, he replied: "We are weighing the detail."
In an article for The Independent, Mr Cook said he "would be surprised" to learn that Britain had "intercepted the calls of Kofi Annan" in the run up to the war.
"I never met anyone in the diplomatic community less likely to engage in subterfuge than Kofi," he said.
"If I, while foreign secretary, wanted to know his honest views, he would courteously and patiently explain them to me if I rang up and asked."
Her former deputy George Foulkes said: "This is part of Clare's political agenda to undermine the prime minister, and it is damaging both to the government and to the party which gave her all the privileges she enjoyed in government.
"She should put up or shut up."
But former UN Secretary General Boutros Boutros-Ghali said he had been warned his office would be bugged.
"From the first day I entered my office they told me: 'Beware, your office is bugged, your residence is bugged and it's a tradition member states who have the technical capacity to bug will do it without hesitation'," he told BBC Radio 4's Today programme.
Mr Boutros-Ghali called for more to be done to protect the UN chief from spying, which he said could cause diplomatic problems and negotiations to fail.
But he added that if he, like Ms Short, had been part of the British Government he would not have spoken out.
That view was echoed by Ms Short's former Cabinet colleagues, including Jack Cunningham, who argued that ministers should "accept and comply with the duty and obligations" and the privileges that came with the position.
"Clare Short obviously has a personal agenda which is to attack, damage and undermine the prime minister at every opportunity and sadly, that's been the hallmark of her conduct and behaviour since she left the government."
Ex-Scotland secretary Helen Liddell said Ms Short's remarks were "completely unsubstantiated" and urged her to keep quiet.
"It is becoming increasingly difficult to work out where Clare is coming from. This is a pattern of behaviour that really is confounding her friends and colleagues and I don't think it is doing the country any good," she said.
UN chief spokesman Fred Eckhard said any bugging would be illegal and should be stopped.
Sir Crispin Tickell, a former British ambassador to the UN, told Today Ms Short had committed a "major breach of confidence".
But he said bugging was not necessarily "a bad thing" if done in the national interest - and he doubted whether her claim had affected diplomatic trust.
Edward Mortimer, Mr Annan's director of communications, said the UN had asked the British Government whether Ms Short's claims were true.
"I don't think we have got an answer precisely on that," he told Today.