Conditions for asylum-seekers are becoming tougher in many countries because of fears of terrorist attacks, the UN refugee agency has warned.
Some refugees manage to return home, like these Afghans
Speaking on World Refugee Day, UNHCR head Antonio Guterres said some nations had curbed immigration to the point where refugees were being excluded.
He told the BBC that refugees were not terrorists, but the victims of terror.
After a five-year fall, the number of refugees is rising again because of violence in Iraq and Somalia.
The UN estimates that nearly 44m people have left their homes because of violence or persecution.
It says that some are forced abroad, many others are displaced within their own countries.
"The international community is not paying attention and is not giving enough support," UN High Commissioner for Refugees Antonio Guterres said.
However, the UN said there were reasons for optimism amid the grim statistics.
Millions of refugees have returned home to countries such as Sudan, Afghanistan and the Democratic Republic of Congo.
Mr Guterres is in southern Sudan, where he seeing the challenges facing refugees returning there after many years of conflict.
He said people were returning home from all neighbouring countries and "showing a lot of courage, a lot of commitment to build their new country".
Mr Guterres said the response among western countries had been mixed.
"There are countries in which, especially after 9/11, there has been a growing concern with refugees and I think it's important to say and to repeat they are not terrorists, they are the first victims of terror," he said.
"But in other countries, on the contrary, we are seeing an extremely generous approach in protection granted to refugees."
He singled out Sweden and the Netherlands for being "extremely positive" in relation to the plight of Iraqi refugees.
Conflict is the biggest factor forcing people from their homes.
The war in Iraq alone has displaced an estimated 4m people - 2m inside and 2m outside the country.