Gen Abdelaziz has promised to hold fresh elections
The United States has suspended more than $20m (£11m) in non-humanitarian aid to Mauritania after a coup.
The US state department said it condemned in the strongest possible terms the overthrow of the country's first democratically-elected president.
But General Mohamed Ould Abdelaziz, who led the military coup, said the army would safeguard democracy.
Meanwhile, the Arab League and the African Union have demanded the ousted president be released immediately.
Diplomats from both organisations are due in Mauritania on Friday to discuss the situation with the coup leaders.
President Sidi Ould Cheikh Abdallahi was detained by renegade soldiers on Wednesday after he tried to dismiss four senior army officers - including Mr Abdelaziz, the head of the presidential guard.
Prime Minister Yahia Ould Ahmed El-Waqef - who the coup leaders had also detained - was reported to have been taken to a barracks near the presidency.
The whereabouts of the president are still unclear, and his daughter, Amal Cheikh Abdallahi, said she did not know where her father was.
"I'm very worried about his health and his security," she told the BBC's Focus on Africa programme.
"He doesn't have the right to move or to call. He doesn't have a phone. He doesn't have liberty," she said.
The US aid suspended includes $15m (£7.5m) in military-to-military co-operation, more than $4m (£2m) in peacekeeping training, and more than $3m (£1.5m) in development assistance.
Some MPs and parties have expressed support for the military intervention
France repeated its condemnation of the coup. A statement from the French president's office warned that France might take measures against the coup leaders, although it did not elaborate further.
Gen Abdelaziz said the new military council, which has promised to hold elections, would "solve the country's problems".
"The armed forces and the security forces will always stay with the people to deepen the democracy," he said in the capital, Nouakchott.
"It's them who brought the democracy here and it's them who have always protected this democracy and they will always preserve it."
On Thursday, there were demonstrations for and against the coup in Nouakchott.
But the BBC's James Copnall, who arrived in the city on Thursday evening, the day after the takeover, says it is remarkably calm and relaxed.
He said some people at the airport were joking about the situation - possibly as it is not regarded as that out of the ordinary given the country's history of coups.
The military has been involved in nearly every government since Mauritania's independence from France in 1960.
Presidential elections held in 2007 ended a two-year period of military rule - the product of a military coup in 2005.
Despite the widespread international condemnation of the takeover, many MPs and political parties have expressed their support for it.
Senate Vice-President Morsen Ould al-Haj said that the president had abused his powers and was particularly angered by the influence his daughter and wife wielded.
"He failed completely - he transformed everything into a family business. He became very stubborn; he started by installing his children all parts of the government," he told the BBC.
"Each of his children consider themselves himself a prince ready to inherit the country. They are a real royal family."