Many refugees are harassed in foreign camps
Human Rights Watch says ethnically-motivated attacks, lawlessness and fighting between rival warlords remain rampant in the country.
Its report comes days before the United Nations is due to open repatriation centres for Afghans living abroad.
The centres would be able to process 5,000 people a day and supply the refugees with food and equipment for resettlement.
However, the report's authors say many Afghans living abroad are too frightened to return home.
And those who grew up and established new homes and communities in their adopted countries may find it a gamble to return to their homeland.
"While many Afghan refugees want to go home, there are others terrified of returning at this time," said Rachel Reilly, refugee policy director at Human Rights Watch.
Climate of fear
Most of the refugees fled the former Taleban regime in Afghanistan; others sought shelter in neighbouring countries after the outbreak of the war.
Among those wanting to return are university graduates and skilled workers who are hoping to obtain jobs in the country's rapidly expanding aid industry.
However the group says there is still a climate of fear in Afghanistan and that, while 140,000 Afghans returned home in the past six weeks, 50,000 others left the country.
It says refugees who are suspected of being Taleban sympathisers are at risk of reprisal attacks inside Afghanistan.
The report quoted one woman as saying: "I am Pashtun and not with the Taleban, but people think I am with them and they will punish me and become angry with me."
The Taleban Government was mainly made up of ethnic Pashtuns.
Host countries accused
Human Rights Watch has also criticised the treatment of the refugees in Pakistan and Iran.
It said many who did not have identification documents were harassed and beaten at border crossings and in camps.
"Many Afghans are in a no-win situation.
"They have endured violence in Afghanistan and many fear going home - but they also live with harassment and violence in Pakistan and Iran," said Rachel Reilly.
The group called on Iran and Pakistan to provide the refugees with the necessary documentation and to bring an end to abuses against them.
It also urged donor countries to help fund Afghanistan's Ministry for the Return of Refugees and for refugees to be allowed to return voluntarily under secure conditions.