Col Gaddafi had wanted to stay on as AU chairman
Libya's leader Muammar Gaddafi has failed in his bid to stay on as president of the African Union for another year.
At the annual AU summit in Ethiopia, leaders from 53 African countries chose the president of Malawi, Bingu wa Mutharika, to take his place.
A BBC correspondent at the summit says Col Gaddafi was very reluctant to stand down, causing considerable resentment.
He used his farewell speech to call for political unity in Africa.
Earlier UN Secretary General Ban Ki-moon urged African leaders to work for national unity in Sudan to prevent the south seceding from the north.
Mr Ban said both the UN and AU had a big responsibility "to maintain peace in Sudan and make unity attractive".
A referendum is due next year on whether the oil-rich south should become independent.
Libya has chaired the AU for the past year, and under the system of rotating regional blocs, the job was due to go to a southern African leader.
However, Mr Gaddafi wanted to extend the term. He had the support of Tunisia, and is said to have won over some smaller countries by paying their AU membership dues.
Malawi was apparently backed solidly for the role by southern and eastern African countries.
Mr wa Mutharika was backed by southern African nations
The BBC's Uduak Amimo says the organisation needed a country chair with strong financial muscle, like Libya, but also needed to be seen to be respecting its own rules and processes.
After conceding the presidency, Mr Gaddafi said he would continue to promote his vision of a "United States of Africa", adding that he did not need to keep the title of AU head.
"My brother president of the Republic of Malawi will replace me and take over," he said.
"There is no need for any title, I'll remain in the front struggling."
The theme of the three-day AU summit in Addis Ababa is information and technology.
In an opening speech to the African leaders, Mr Ban called for renewed efforts to meet the Millennium Development Goals, which include reducing poverty, disease and child mortality, ahead of their target date of 2015.
"We have seen a sharp decrease in malaria and measles deaths across the continent, virtual gains in primary school enrolment, marked improvement in child health," he said.
"We must build on these successes and help spread them around the world."
Heads of states will also be discussing, among other issues, the escalating violence in Somalia.
Sudanese President Omar al-Bashir has said he will accept the result of a referendum even if the south voted for independence.
"Whatever the result of the [southern Sudan's] referendum we have to think how to manage the outcome," Mr Ban said in a joint interview with AFP and RFI radio.
"It is very important for Sudan but also for the region. We'll work hard to avoid a possible secession," he added.
Sudan's mainly Muslim north and the animist and Christian south ended a two-decade war in 2005 and joined a unity government.
But tensions remain high ahead of the country's first genuine multi-party national elections since 1986, due in April.
The south, which has a semi-autonomous government, is likely to vote to secede from the north in the 2011 referendum, correspondents say.