Hutu rebels opposed to Burundi's power-sharing government have been meeting senior Tutsi government figures at a secret location in Nairobi.
President Ndayizeye, a Hutu, has not been able to end the war
The first talks between the two sides began on Saturday, although the rebels refuse to recognise the government.
On Sunday, two rebels belonging to the main Hutu group arrived in the capital to take up their ministerial posts.
Some 300,000 people died in a brutal 10-year civil war between Hutu rebels and the Tutsi-dominated army.
In the Kenyan capital, a spokesman for the National Liberation Forces said they did not recognise the legitimacy of the government, and were meeting officials and military leaders merely in their capacity as members of the Tutsi minority.
"We are not negotiating with the government of President Domitien Ndayizeye, but with Tutsi delegates," FNL spokesman Pasteur Habimana told AFP news agency.
But in Bujumbura, the new FDD ministers said they were happy to return home from exile under the terms of a peace agreement reached with the FDD earlier this month.
The new interior minister, Simon Nyandwi, said it was important Burundians moved away from what had divided them.
4 FDD ministers
40% of army officers
Second assembly vice-president
Assembly deputy secretary general
35% of a new police force
35% of vacant secret service posts
FDD fighters to be demobilised
New communications minister, Onesime Nduwimana, said he wanted to restore confidence between the authorities and journalists.
Several more senior FDD officials are expected in Bujumbura by the end of the week, including FDD leader Pierre Nkurunziza.
With the arrival of the FDD ministers, optimism is growing that long-running efforts to secure peace may finally be paying off.