The BBC correspondent in Burundi says that all was quiet in Bujumbura, with the inhabitants of the city going about their business as if nothing had happened.
In his first comments on the failed coup, President Buyoya said he was surprised that such action was taken when the peace process had gone so far.
He also praised residents in the capital for remaining calm.
Burundian Information Minister Luc Rukingama told the BBC that he suspected some politicians were probably behind the coup attempt, and there would be an inquiry.
The Defence Minister, Cyrille Ndayirukiye, said that all the soldiers who had taken over the radio on Wednesday - including the officer who led the coup - had given themselves up peacefully and were in custody.
No-one had been killed or wounded, Mr Ndayirukiye said.
The 30 dissident officers from the Tutsi-dominated army were opposed to President Pierre Buyoya's negotiations with Hutu rebels.
Unknown group
The radio station was seized at 1630 (1430 GMT) on Wednesday by a group calling itself the Patriotic Youth Front.
After the guard fled, the soldiers played a tape announcing President Buyoya's removal from office, the dissolution of the National Assembly and the closure of the airport.
The statement was attributed to Lieutenant Gaston Ntakarutimana, a commander at the Gakumbo military camp, which is responsible for protecting the airport.
The group had never been heard of before.
The radio station was surrounded by forces loyal to the government - but the army decided not to storm it, opting instead to wait until the rebels gave themselves up.
Our correspondent in the region says one theory is that the rebels were supposed to have been joined by a second larger group who, in fact, failed to go ahead with the plan.
She also says there is speculation in Bujumbura, that the coup attempt may have been stage-managed to boost the waning popularity of President Buyoya.
Peace talks
Mr Buyoya was in Libreville, Gabon, at the time for peace talks with the leader of the main rebel group fighting the government in the country's civil war.
He was in a meeting with South African Deputy President Jacob Zuma, when news of the coup was broadcast, a spokesman for Mr Zuma said.
More than 200,000 people, mostly civilians, have been killed in Burundi since 1993, when civil war broke out after soldiers from the Tutsi minority killed the first democratically -elected president, a Hutu.
A peace accord was signed last August in Arusha, Tanzania, by 19 parties involved in the conflict, but it was rejected by the two rebel movements and fighting has continued.