"The first and immediate priority issue that we all agreed was that we must feed the hungry," Mr Ban said after a meeting of agency heads in the Swiss capital, Bern.
"Without full funding of these emergency requirements, we risk again the spectre of widespread hunger, malnutrition and social unrest on an unprecedented scale."
Mr Ban said it was essential to support farmers in poor countries who were producing less because of the high cost of fertilizer and energy, and to this end he said the task force hoped to:
offer $200m financial support to farmers in the worst affected countries to boost food production
set up a $1.7bn programme to help countries with a food deficit to buy seeds
For its part the World Bank said that it would:
double its lending for agriculture in Africa over the next year
consider providing quicker and more flexible financing for poor countries
Mr Ban said Africa could double its production of food in a few years with an annual investment of $8-10bn.
The BBC's Alastair Leithead visits a World Food Programme service near Kabul
He also called on the international community to "urgently address trade-distorting subsidies in developed countries, and the ongoing Doha trade round.
"But also in the long term we need to address the challenges caused by climate change," Mr Ban added.
The head of the World Bank, Robert Zoellick, who also attended the meeting in Bern, urged countries not to use export bans to protect food stocks.
"These controls encourage hoarding, drive up prices and hurt the poorest people around the world who are struggling to feed themselves," he said.
His comments came as India decided to tax exports of basmati rice as it tries to control domestic inflation, Reuters news agency said.
India banned exports of non-basmati rice in March.
Renting fields
The prices of staple foods including rice, grain, oil and sugar are all at least 50% higher than they were this time last year.
On Tuesday the Beijing Morning newspaper reported that China might lease fields in Latin America, Australia and the former Soviet Union to replace farmland lost to urban and industrial development.
Meanwhile in Washington, US President George W Bush said he was "deeply concerned" by high food prices at home and abroad.
He said that diverting corn for the production of biofuels had only accounted for 15% of the rise in prices, which had otherwise been caused by weather, energy prices and increased demand.
"It's in our national interest that we - our farmers - grow energy, as opposed to us purchasing energy from parts of the world that are unstable or may not like us," he added.
Mr Bush said the long-term solution would be to switch to cellulosic ethanol, which uses grasses or other non-food sources to produce fuels.
He also said he had made a proposal to Congress on buying food from local farmers so that countries could become "self-sustaining".
Bookmark with:
What are these?