Trade among African countries accounts for only about 10% of their total exports and imports, according to a report.
African integration remains elusive
The study by the Economic Commission for Africa (ECA) blames the continuing low level of trade on poor transport links among African countries.
Instead it suggests that colonial-era patterns remain, with most trade still to and from the former colonial powers.
It says Africa has a long way to go before reaching EU-style integration.
As a result the full potential of a vast market has clearly not been realised, it said.
"Many of the countries are small so we need to have regional integration, larger markets," said the ECA's executive secretary, K.Y. Amoako.
The report makes many recommendations on how to accelerate the integration process.
It says there is a need for a greater role by regional bodies like the Africa Union and the Regional Economic Communities.
Such groups need to receive more financial and human capital, plus supranational authority, it says.
Also regional transport costs need to be consolidated. Such costs are prohibitive because many national road, air and rail networks remain unconnected within Africa.
As a result the report says, for example, that shipping a car from Japan to Abidjan in Ivory Coast costs $1,500 (£806). But shipping a car from Addis Ababa, Ethiopia, to Abidjan would cost $5,000,(£2,698).
Africa's telecommunications have made more progress though. The report says this is due to the global revolution in technology and the widespread privatization of phone companies.
Mobile phones are in wide use everywhere across the continent except central Africa.
Africa is the only region of the developing world which is no better off than it was 25 years ago.
The Commission for Africa was set up in February 2004 by British Prime Minister Tony Blair to analyse Africa's problems and submit potential solutions.
It aims to look at economic issues, education, conflict resolution, health, the environment, the HIV Aids and governance.
South African finance minister Trevor Manuel, UK chancellor Gordon Brown and musician and human rights activist Bob Geldof are among those on the commission.