Ruud Lubbers was speaking during a visit to camps housing 60,000 displaced Afghans on the border with Pakistan.
"It's a combination of three very vulnerable groups," Mr Lubbers told reporters in Spin Boldak, where two of the camps are situated.
Most of the refugees are Pashtuns from the north and west who moved south to escape persecution by Tajiks and Uzbeks after the collapse of the Taleban.
The rest are nomadic shepherds and southern Pashtuns left with nothing after four years of drought.
Mr Lubbers said that helping the refugees to go home required much more than just humanitarian aid.
"It's not only a question of money and it's not only a question of UNHCR."
He said it depended on the country's leaders stopping factional fighting going on throughout much of the country.
"This is not just about assistance, but a new life and the possibility of going home.
"As we've heard today, that's complicated, but we'll go for that and fulfil our commitment."
Refugee flood
Nearly half of Afghanistan's one million "internal displaced" live in the south of the country.
"We are just waiting. If the situation in the north is peaceful, we'll return home. But right now, it hasn't improved, so we'll stay here," Mohammed Hanif, spokesman for up to 200 families from the north, told Mr Lubbers.
Resources have already been put under strain by huge numbers of Afghans flooding back this year from camps in Pakistan and Iran.
It is estimated that up to 1.5 million refugees have returned out of four million living abroad - far more than the 400,000 predicted in 2002.
Mr Lubbers is due to hold talks on Tuesday with UN envoy Lakhdar Brahimi in Kabul before returning to Geneva the next day.
He will report back to donors next month - stressing that Afghan returnees need long term support to help them reintegrate and ensure they do not flee the country again.