MPs from the main political parties have accepted that they need to face up to the electoral challenge posed by the British National Party.
The BNP says the report highlights a sense of "frustration"
Research carried out for the Joseph Rowntree Charitable Trust has suggested that up to 25% of people have thought about voting for the far-right party.
Both the Tories and the Liberal Democrats say disillusionment with government institutions is to blame.
But ministers say any increase in BNP backing would just be a "protest vote".
Home Office minister Andy Burnham dismissed the likelihood of the BNP becoming a stronger electoral force.
"They pose a very localised threat and I am worried that if we give them too much coverage, it can back up the notion that they are a potent protest vote."
However the Rowntree report echoes comments by Employment Minister Margaret Hodge, who said voters may be tempted by the BNP, in May's local elections in England next month.
BNP COUNCIL SEATS
First council seat gained in Millwall, south-east London, in 1993
Won three seats in Burnley, Lancashire, in 2002
By 2004 had 17 seats
Currently has 15 seats, including six in Burnley
Former Conservative leader Iain Duncan Smith, who is chairing the party's social justice policy group, said he was not surprised to see the BNP's message gaining purchase in disadvantaged areas.
"What we have picked up in these very difficult communities is the collapse in the quality of life for so many people," he told BBC Radio 4's Today programme.
Communities were frustrated over "things like poor housing, inability to get work, the collapse of manual labour, no educational training, anti-social behaviour on the streets, the cleanliness of the street", said Mr Duncan Smith.
"The whole sense of the quality of life in these communities has become a rich feeding ground for people who want to stigmatise others as being the cause of this."
He went on: "They now feel - and this is the bit that worries me - that there is a bigger gap between the government and the governed in these areas than there has been in modern times."
Liberal Democrat President Simon Hughes urged voters not to be taken in by the "simplistic promises" of the BNP, and said that the main parties had only themselves to blame if people were turning away from them.
"If voters are unhappy with conventional parties, one of the key reasons is because successive Tory and Labour governments have failed to provide enough affordable housing where families wish to live," he said.
The authors of the research asked focus groups about their voting views and looked at a series of opinion polls that asked people which party they might consider voting for.
Professor Peter John of Manchester University, said the report showed "underlying support" for the BNP rather than voting intentions
"It is not what party you will vote for, but who you might vote for," he told BBC Radio 4's Today programme.
Prof John also said the far-right party tended to have more support in predominantly white, working class areas.
"They feel their voices have not been heard," he said.
The BNP has courted controversy over its policies, which include a total ban on immigration, and the forced deportation of illegal immigrants from the UK.
In the 2005 general election, the party raised its total number of votes by 0.5% to gain 0.7% - or 192,850 votes.
BNP spokesman Phil Edwards said the Rowntree report reflected unease among voters about Britain's shift towards a multi-cultural society.
"That does add quite a lot of tensions and stresses," Mr Edwards said.
"What we are trying to do is preserve the traditional culture and identity of Britain," he added.
The BNP has said it is putting up more candidates than ever before - 356 - for May's local elections.
Operation Black Vote, a group which campaigns to make politics more multi-racial, agreed that voters in run-down areas felt the government had let them down.
"In these areas, deprivation and poverty exist," Simon Woolley, national co-ordinator for Operation Black Vote, told BBC Radio Five Live.
"Now that's a genuine debate to be had. ... It's nothing to do with black people."
The report, prepared by the Democratic Audit and funded by the Joseph Rowntree Charitable Trust, social policy research group, is due to be published on April 25.