Ban Ki-moon visited Burma after Cyclone Nargis in May
UN Secretary General Ban Ki-moon has expressed frustration at the lack of democratic reform in Burma and says now is not the time for another visit.
In a letter to Mr Ban, 112 ex-leaders of various countries urged him to go there to push for the release of Burma's political prisoners.
He said he would go when there were "reasonable expectations" a visit would be "productive and meaningful".
Burma has arrested hundreds of dissidents this year.
The letter, sent on Wednesday, was signed by ex-leaders including ex-US presidents George Bush and Jimmy Carter, and former UK prime ministers Tony Blair and Margaret Thatcher.
It called for the release of 2,100 political prisoners.
It said: "We urge you to make it clear that all political prisoners in Burma must be released by the end of this year, regardless of whether you travel to Burma.
"If the Burmese junta continues to defy the United Nations by refusing to make these releases by the end of the year, we urge you to encourage the Security Council to take further concrete action to implement its call for the release of all political prisoners."
But Mr Ban, speaking to reporters at the United Nations on Friday, said: "As I have said repeatedly, I am ready to visit Myanmar [Burma] again, to continue our consultations on various issues - humanitarian issues, and also political issues.
"At this time I do not think that the atmosphere is ripe for me to undertake my own visit there.
"But I am committed, and I am ready to visit any time, whenever I can have reasonable expectations of my visit, to be productive and meaningful.
The pressure on Mr Ban was led by Norway's former prime minister Kjell Magne Bondevik, who wanted him to enforce the Security Council's call in October 2007 for Burma to release the prisoners, including Nobel Peace Prize winner Aung San Suu Kyi.
It was released by Mr Bondevik's Oslo Center for Peace and Human Rights and by a Washington area-based advocacy group, Freedom Now.
Mr Ban did visit Burma in May after Cyclone Nargis hit.
Burma has been criticised by human rights groups for its suppression of pro-democracy activists last year.
The UN issued a statement calling for democratic reforms following last year's protests, but Burma has continued its crackdown. Last month some dissidents were sentenced to 65 years in jail.
The most prominent political prisoner is National League for Democracy leader Ms Suu Kyi, who has been in jail or under house arrest for most of the last 19 years.