Both countries accuse the other of harbouring and training rebel fighters hostile to the other's government.
The meeting, in the Rwanda capital Kigali, was the first of a joint military commission set up by President Paul Kagame and his Ugandan counterpart, Yoweri Museveni.
The two men are to meet in London on 6 November, at a summit organised by UK International Development Secretary Clare Short.
Secretary General of the Organisation for African Unity Amara Essy arrived in Kigali on Saturday for talks with President Kagame.
Congo clashes
The aim of the military commission is to allow each party to investigate allegations of preparation for war by the other.
There are also plans for a meeting between the two countries' chiefs of staff on the Rwanda-Uganda border in the coming days.
President Museveni has accused Rwanda of organising "terrorism" against his country, according to a letter he wrote to Ms Short, which was published by the Rwandan press.
It is now more than two years since disagreements over how the war in Democratic Republic of Congo was being conducted degenerated into full scale fighting between the Rwandan and Ugandan armies when they were occupying the Congolese jungle town of Kisangani.
Rwanda emerged victorious from the fighting, and since then relations between the two countries, which are both still embroiled in the Congo war, have gone downhill fast.
Uganda has in recent weeks stepped up hostile declarations against Rwanda.
Ties between Rwanda and Uganda have hit a new low despite efforts at reconciliation between the two former allies.
Mr Museveni and Mr Kagame met in July, pledging to improve relations which have been soured by their involvement in the conflict in the Democratic Republic of Congo.
But the mistrust has returned amid mutual accusations that the other country is backing dissidents fighting their respective governments.
Falling out
"We think the situation will not degenerate into full scale war," Reuters quoted a western diplomat in Kampala as saying.
"But we are concerned that the two countries will take a long time to get back to a cordial relationship," he said.
Uganda and Rwanda were once the firmest of allies, with Uganda supporting Mr Kagame when his then rebel army fought its way to power in 1994.
However, they fell out two years ago, soon after both countries decided to back rebels fighting the government in the Democratic Republic of Congo.
Despite being on the same side, rival political and business interests led to a series of violent clashes between their armies in the north-eastern Congolese city of Kisangani.
Relations between the two countries have never recovered.