Southern and East African leaders are split over plans for a pan-African government, as suggested by Libya's head of state Col Muammar Gaddafi.
Not all the African leaders at the summit see eye to eye on unity
Uganda's Yoweri Museveni said he backed economic integration but said Africa was too diverse for one government.
"Politically we should only integrate with people who are either similar or compatible with us," he said, according to Uganda's state-owned media.
Senegal, however, backed the plans and said a breakaway group could be formed.
On the final day of the African Union (AU) summit, the BBC's Will Ross in the Ghanaian capital, Accra, says there are clear differences of opinion over the degree of integration and the speed.
Ghana's Foreign Minister Nana Akufo-Addo believes such problems were inevitable but can be overcome.
"You know the problems that you have in the European Union with 25 members, now 27, to arrive at common positions - we have 53," he said.
"So clearly there'll be problems involved for people to adjust and I believe that the 53 states will find a way of sharing and joining in the consensus as to the future direction of our continental organisation."
Senegal, one of Africa's most stable democracies, is backing Mr Gaddafi's call for the immediate set up of a pan-African government.
"We are ready to abandon partially or totally our sovereignty to join a unity government in Africa. So we have no problem. My president is here with his pen ready to sign," Senegalese Foreign Minister Cheikh Tidiane Gadio said.
He suggested a small group of states could sign up to a federation now and wait for others to follow.
The leaders of Kenya and Lesotho, representing southern Africa, also expressed their doubts.
"We recognise that Africa's interests would be best served through economic and political integration," AFP news agency quotes Lesotho's Prime Minister Bethuel Pakalitha Mosisil as saying.
"However we must adopt a bottom-up approach, not a top-down one - We believe that such integration should be gradual rather than precipitous."
Our correspondent says the majority of African leaders are likely to call for a gradual approach, preferring to strengthen the existing regional blocs rather than signing away some of their own sovereignty.
'Take the bull by the horns'
The idea of a single pan-African government was first promoted by Kwame Nkrumah, who led Ghana to independence in 1957.
On Monday, Zimbabwean President Robert Mugabe said unity was vital to make the continent truly independent of the West, as he spoke to a crowd of cheering Ghanaians.
Mr Gaddafi has called for the immediate establishment of a single government, foreign policy and army.
Ghana's President John Kufuor said in his opening speech to the conference that the question of unifying Africa was not in doubt, but the key issue was how to attain it.
AU Commission head Alpha Oumar Konare told the gathering that Africans needed to "take the bull by the horns and move towards a new country - Africa".
But campaigners on the sidelines of the summit say delivery is the key problem, with leaders already having shown they are unwilling to give up power to regional economic blocs.
"We have regional economy communities that were put in place for West Africa... but nothing is working. From one country to another... there are still a lot of obstacles," a campaigner for the organisation Call To Action Against Poverty told the BBC.
This summit is the ninth since the AU was created five years ago.