Promises of aid to Africa will be checked in a new annual review, the Africa Partnership Forum has agreed.
Britain is giving £200m to a new fund for Africa
Members have met in London to discuss how to ensure pledges made for improving the continent are met.
The report will examine what aid has been handed over and give updates on development goals and African governments' promises to make changes.
The forum - involving G8 and African nations - agreed the first edition would be published in October 2006.
Oxfam said this was a "crucial phase" after the commitments made this year.
The forum agreed to draw up a joint action plan bringing together pledges made by donor countries and African recipients, to help the group monitor the delivery of pledges.
It refers to promises in recent months to boost aid substantially and cancel debts, as well as pledges made by African nations to make good use of aid in reducing poverty and eliminating corruption.
A support unit, based at the Organisation of Economic Co-operation and Development (OECD) in Paris, will be created to help with the task.
The UK's International Development Secretary Hilary Benn said the forum had designed a way to answer the question many have asked after the commitments made to Africa this year.
"How can we be sure that they are going be implemented, how can we be sure that people are going to do the things that they promised to do?
"The Africa Partnership Forum has now been charged with that task."
Mr Benn said 2005 had seen unprecedented commitment to African development, with July's G8 meeting promising up to $55bn (£31.2bn) worth of debt relief and EU states pledging a doubling of aid by 2010.
To coincide with the forum, Mr Benn put £200m into a new World Bank fund designed to speed up progress in reducing preventable disease, getting more girls into school and giving more access to clean water in Africa.
On whether the forum would sanction those who did not fulfil their promises, he said: "It's not a question of compelling or censuring. It's each of us taking on the responsibility to do the things that we said we were going to do.
"In the end, what will call all of us to account is pressure from within our own countries, the normal operation of democracy, the cut and thrust of politics."
Nigeria's finance minister, Ngozi Okonjo-Iweala, said there had been "a lot of talking" and commitments from all sides of the international community.
"Now it is time to focus on implementation and outcomes," she said.
Oxfam spokeswoman Sarah Kline said: "Everyone was watching when the pledges of aid were made at Gleneagles, but the crucial phase is now.
There is pressure on all leaders to get it right on tackling poverty
"Promises are one thing but now we need to see follow-through on commitments made by the G8 governments."
She added: "While it is vital that we do not lose track of money that has been pledged, attention will now focus on trade reform."
She said decisions taken at the World Trade Organisation meeting in Hong Kong, in December, would ultimately decide whether impoverished countries were to be allowed to "trade their way to prosperity".
The Africa Partnership Forum involves representatives of the G8, OECD, African Union, the New Partnership for Africa's Development and international institutions like the UN, World Bank and International Monetary Fund.