Sudanese women who escaped the Darfur conflict to eastern Chad are facing high levels of sexual violence, an Amnesty International report says.
Despite the presence of a UN force, women and girls are being attacked when they leave 12 designated camps in search of water, the report says.
It also documents cases of refugees being attacked inside the camps by Chadian aid workers.
Chad's government has denied that any Chadian has attacked a Sudan refugee.
Since 2003 about 250,000 Darfuris have fled the conflict in Sudan, where mass rape of civilians had allegedly been used as as strategy to displace entire villages.
"The rape that countless women and girls experienced in Darfur continues to haunt them in eastern Chad," Tawanda Hondora, Amnesty's Africa programme deputy director, said in a statement.
The rights group says in many cases women are too scared to report the abuse, or attackers escape without being brought to justice.
The BBC's Celeste Hicks in Chad says life is tough for the refugees, who scratch out a living from the bleak sands of the Sahel.
In temperatures well in excess of 40C (104F), most women are forced to leave the camps to look for extra water and wood for cooking.
Amnesty claims this is when they are most at risk of attack as many local Chadians resent the fact that Sudanese get fuel, food, water and medicines.
One Sudanese refugee, Mariam, told the BBC that people from Djabal refugee camp near Goz Beida had come into conflict with Chadian villagers.
She recounted one incident where nine women went into a village to collect wood.
"They were stopped by some men from the village," she said.
"[The men] took their materials and attacked [them] with sticks and stones. Now the children are too scared to go out alone."
Mr Hondora said women also face danger inside camps.
"They face the risk of rape and other violence at the hands of family members, other refugees and staff of humanitarian organisations, whose task it is to provide them with assistance and support," he said.
Many of the women said they were worried about being abandoned by their families or shunned by the community if the attacks came to light.
"There is no security, the UN peacekeeping force in Chad is not providing that security and neither is the Chadian armed forces," Mr Hondora told the BBC's World Today programme.
Our correspondent says the UN peacekeeping mission, Minurcat which provides security patrols in the east is suffering from lack of funds.
Currently it has only about half of its mandated soldiers.
A special UN-trained Chadian police unit, the DIS, is supposed to investigate rape cases, but many women complained that they were not taken seriously.
"The DIS spends a lot of time protecting themselves. Even the UN soldiers have to protect them. No-one seems to have much time to protect us," a woman at Gaga Refugee Camp told Amnesty.
Minurcat's Michel Bonnardeaux says part of the problem is that weak policing and judicial capacity means cases are not prosecuted.
Few of the 278 people arrested by the DIS in 2009 have been brought to trial, our reporter says.