Goverment policies have been blamed for fuelling discrimination against people with HIV.
Many people with HIV keep their condition quiet
A study by the National Aids Trust and Sigma Research found gay men and African people living with HIV in the UK face widespread discrimination.
It said negativity can come from employers, families and communities.
But the report also called for a rethink on government policies on asylum and immigration, which it says exacerbate the difficulties.
It also called for the Crown Prosecution Service to rethink its policy of prosecuting people for reckless HIV transmission.
Criminalising people in this way "profoundly reinforces stigma and discrimination related to HIV", the report said, and left people uncertain about the legality of their actions.
Home Office criticism
The report criticised the Home Office policy of dispersing asylum seekers with HIV across the country for potentially denying people social support, and reducing their access to specialist care.
And it branded a ban on allowing asylum seekers from seeking legal employment as "harmful" as it damaged people's ability to support themselves.
The report recommended that the government take action to make it easier for all people living with HIV to work, including education for employers about HIV, flexible working and job sharing.
The Department of Health is urged to rethink the current policy of charging asylum seekers, whose application has failed, for non-urgent hospital care.
Journalists, too, are blamed for inaccurate and stigmatising coverage of HIV issues.
Deborah Jack, chief executive of the National AIDS Trust, said: "This research highlights the continuing stigma associated with HIV and the discrimination faced by people living with the virus.
"Action is urgently needed by the government, communities and HIV organisations to break down this stigma which has consequences for both individuals and for public health."
The report found fear of discrimination often prevented gay men disclosing their HIV status to family members.
Similarly, fear of dismissal prevents many people disclosing HIV status to employers.
It also found HIV is often a taboo in African communities living in the UK. As a result many African people with HIV fail to get access to adequate treatment and care.
The study was based on interviews with 150 people living with HIV.
Lisa Power, Head of Policy at Terrence Higgins Trust, said she fully agreed with the report's conclusions - and said the experiences chimed closely with those who used the charity's services.
"People tell us that HIV specific discrimination is frequently compounded by racism or homophobia or sometimes both, and sometimes the worst betrayal for someone newly diagnosed with HIV can come from the disadvantaged community they were already part of."
A Home Office spokesperson defended current policy, and said 80% of asylum applications were decided within two months.
Although asylum seekers were dispersed to try to avoid the South East taking a disproportionate share, people were only moved to areas where they could access proper medical support, and their social care needs were assessed by an outreach team.