Some are wary of the role of Libya's Colonel Gaddafi in the new organisation - formed less than two years after he first suggested a "United States of Africa" at a special summit in Sirte, Libya.
The continent of Africa consists of a jigsaw of countries overlaying another jigsaw of ethnic allegiances.
Since decolonisation began in earnest nearly 50 years ago, these fragmented communities have sought to create a common bond through the Organisation of African Unity.
Critics say that throughout its 38 years the OAU has merely been a talking shop and a waste of money, while its supporters maintain that it has achieved its primary goal of African liberation.
When it was established in 1963, some African leaders wanted continental unity straight away while others argued that the newly independent countries had to establish strong foundations first.
There is general agreement that the OAU's proudest moments came in the fight against colonialism.
In the 1970s, it organised material and moral support for the liberation movements in countries such as Zimbabwe and Mozambique.
On the negative side, the concept of non-interference in internal disputes enshrined in the OAU constitution meant that dictators such as Zaire's Mobutu Sese Seko never faced serious pressure from their neighbours to safeguard the human rights of their citizens.
Tajudeen Abdul Raheem of the Uganda-based Pan African Movement, says it was once a "trade union for dictators" but that it has progressed in recent years to the stage that in Lusaka the majority of summiteers will be elected.
Limited power
The OAU has also been criticised for doing little or nothing to improve living standards in the world's poorest continent.
But like any international institution, it can only be as strong as its member states allow it to be.
Those that rule those states have jealously guarded against outside interference that threatens their dominance.
Consequently, the OAU could only pay a very limited role in quelling Africa's many political conflicts, or even in helping to integrate African economies.
The annual summits have often merely provided an excuse to give a face-lift to the host cities, while residents had to put up with the non-stop wailing of sirens, as up to 40 presidents made their way from 5-star hotels to 5-star conference centres.
Outgoing OAU Chairman, Togo's Gnassingbe Eyadema firmly believes that the continent must co-operate more closely, telling the news agency AFP: "The United States of Africa? ... We don't have a choice."
New union
The African Union, based in Addis Ababa like the body it replaces, will aim to unify the 53 African member states politically, socially and economically and will be loosely modelled on the European Union.
Some are wary of the role of Libya's Colonel Gaddafi in the new organisation - formed less than two years after he first suggested a "United States of Africa" at a special summit in Sirte, Libya.
This fear was dismissed by OAU Secretary-General Dr Salim Ahmed Salim when he announced the creation of the African Union in May.
Dr Salim says that it would bring real change to the continent and have an impact on ordinary Africans.
After a year of transition, the African Union hopes to set up a parliament and court of justice next year.
People on the streets of cities from Cape Town to Cairo will be waiting to see what difference these can make to their everyday lives.