Hazel Blears said changes in communities could generate unease
White working class people on some estates feel their concerns about immigration are ignored, Communities Secretary Hazel Blears has said.
She called for a greater effort to challenge "myths" from the far right.
It follows a report suggesting many white people on estates in England feel the government has abandoned them.
For the Conservatives, Baroness Warsi called for an end to policies based on "special needs identity" and a focus on "real core problems".
The study involved interviews of 43 people in Birmingham, Milton Keynes, Norfolk and Cheshire.
The interviews took place on four, predominantly white, housing estates and found people felt a sense of resentment, unfairness and betrayal.
The report said a lack of discussion about concerns had created an atmosphere where rumours spread by the far-right were soon believed to be true.
Ms Blears told the BBC the research showed it was important to get the debate out in the open, to stop the far right from "peddling myths" such as suggestions that immigrants got priority in social housing.
This should be a call to focus on the real core problems of worklessness, debt, welfare dependency, family breakdown and drug and alcohol abuse
Baroness Warsi Shadow social cohesion minister
"The vast majority of people who come to this country have to go into the private rented sector, they don't get council housing" she said.
"There are an awful lot of myths about that people can come into this country, they can get a council house, they get grants for thousands of pounds - a lot of that is simply not true."
But she said, in some parts of the country housing allocation policies were "not as transparent as they should be" and politicians had to be "visible" to address the issues.
The report found it was on matters of housing allocation, that white people felt they were most discriminated against.
Facts which it also highlighted included:
• There was a link between deprivation and apparent hostility to minorities. People who have the least are more likely to be afraid of things being taken away from them
• Few of the people questioned had regular contact with ethnic minorities
• People did not understand integration and thought it was about migrants becoming "like us"
• Respondents found it difficult to talk about their concerns openly for fear of offending or being criticised as racist
Tracey Phillips, a member of the National Community Forum, oversaw the report. She told the BBC: "People told us that they feel they can't say, talk about the things that really bother them for fear of being accused of being racist and that political correction stops any kind of discussions.
"And as a result of that, myths build up and stories come about about how resources are allocated and that's where we get stories about queue jumping."
Hazel Blears: 'There are an awful lot of myths about'
The report also found some felt resentment towards people who were better off than them.
Ms Blears told the BBC: "Where people are struggling and they've got very little, it's not surprising that those people are the people who feel sometimes threatened by change."
In areas which have seen regeneration, people felt less resentful, she said.
But she warned white working-class people "sometimes just don't feel anyone is listening" and have an "acute" fear of the impact of migration.
Baroness Warsi, the Conservatives' community cohesion spokeswoman, said Labour had "completely lost touch with their so-called roots".
"The danger for the rest of us is that this has now created a ticking time bomb of racial and class prejudice," she said.
She added: "I do hope they take the right lessons from this and not use it as an excuse to go down the line of a new 'white relations industry' now to be built on yet another 'special needs identity' .
"This should be a call to focus on the real core problems of worklessness, debt, welfare dependency, family breakdown and drug and alcohol abuse."
Communities ministers are to hold a seminar with other government departments, councils already tackling this issue and leading academics.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.