Al-Shabab and its allies control much of southern Somalia
A second French hostage has been handed over to the hard-line Somali Islamist group, al-Shabab, government sources have told the BBC.
"If they are in the hands of al-Shabab it is very, very serious," said a source in the Somali presidency. The group carries out public executions.
The first man was reportedly given to al-Shabab on Thursday.
The two security advisers, who were training government troops, were seized from a Mogadishu hotel on Tuesday.
BBC East Africa correspondent Will Ross says that, unlike other insurgent groups in Somalia, al-Shabab is unlikely to be holding the men for ransom.
In its eyes, the pair would be enemies, he says.
"They could kill them, saying they are Christian, not Islamic and they could manipulate the situation for their own political demands, including their call for African Union troops to leave," the presidential source told the BBC.
He said the government was not able to negotiate directly with al-Shabab but had been talking to people claiming to be linked to the group holding the two French men.
Al-Shabab and its ally Hizbul-Islam are fighting the UN-backed interim government and together control much of southern Somalia.
Both groups are said to have links to al-Qaeda and have been reinforced by foreign fighters.
A group of gunmen dressed in military uniform seized the men on Tuesday morning and handed them over to Hizbul-Islam.
The move apparently sparked a row with al-Shabab, which now seems to have persuaded the other group to hand the two men over.
BBC Somali Service editor Yusuf Garaad Omar says al-Shabab is known for being the more radical of the two groups.
He says al-Shabab cares little for its public image and has carried out killings on camera.
Somali Prime Minister Omar Abdirashid Ali Sharmarke on Thursday warned Hizbul-Islam it would "bear responsibility for any harmful action taken against the hostages".
The French advisers were helping to train the forces of the government, which has recently appealed for foreign help to tackle the Islamists.
The US last month confirmed that it has sent weapons to the government, which is also being protected by some 4,000 African Union troops in Mogadishu.
Somalia has not had a functioning national government since 1991.
Moderate Islamist Sheikh Sharif Sheikh Ahmed was sworn in as president in January after UN-brokered peace talks.
He promised to introduce Sharia law but the hardliners accuse him of being a western stooge.