A minister who warned about birth defects among children of first cousin marriages in Britain's Asian community has sparked anger among critics.
Mr Woolas warned the problem was "the elephant in the room"
Phil Woolas said health workers were aware such marriages were creating increased risk of genetic problems.
The claims infuriated the Muslim Public Affairs Committee (MPAC) which called on the prime minister to "sack him".
MPAC spokesman Asghar Bukhari said Mr Woolas' comments "verged on Islamophobia".
Mr Woolas, an environment minister who represents ethnically-diverse Oldham East and Saddleworth, risked sparking a major row after warning the issue was "the elephant in the room", Mr Bukhari said.
Mr Woolas said cultural sensitivities made the issue of birth defects difficult to address.
The former race relations minister told the Sunday Times: "If you have a child with your cousin the likelihood is there'll be a genetic problem.
"The issue we need to debate is first cousin marriages, whereby a lot of arranged marriages are with first cousins, and that produces lots of genetic problems in terms of disability [in children]."
Mr Woolas stressed the marriages, which are legal in the UK, were a cultural, not a religious, issue and confined mainly to families originating in rural Pakistan.
But he also told the paper: "If you talk to any primary care worker they will tell you that levels of disability among the... Pakistani population are higher than the general population. And everybody knows it's caused by first cousin marriage."
"Awareness does need to be raised but we are very aware of the sensitivities," he added, pointing out that many of the people involved were the products of such marriages.
His comments come at a sensitive time for community relations following the furore over the Archbishop of Canterbury's comments about accommodating aspects of Sharia law in Britain.
Mr Bukhari told BBC News: "After his comments, MPAC UK is asking will Prime Minister Gordon Brown back him or sack him.
Mr Bukhari said it was "bizarre" Mr Woolas has spoken about a sensitive health issue which has no relation to his environment brief, and accused him of ignoring links between pollution and birth defects.
However, Mr Woolas was defended by cabinet minister Geoff Hoon who said expert analysis was needed on the extent of the problem.
"But it obviously is a very sensitive matter and no one, no one, would suggest this is a problem for the wider Muslim community," he told Sky News.
"I am confident that what he has said will have been said with sensitivity and with proper regard to his Muslim constituents and Muslims right across the United Kingdom."
The call for action was also supported by Labour MP Ann Cryer who raised the issue two years ago after research showed British Pakistanis were 13 times more likely to have children with recessive disorders than the general population.
Mrs Cryer, who represents Keighley in West Yorkshire, told the Sunday Times: "This is to do with a medieval culture where you keep wealth within the family."
"I have encountered cases of blindness and deafness. There was one poor girl who had to have an oxygen tank on her back and breathe from a hole in the front of her neck," she added.
"The parents were warned they should not have any more children. But when the husband returned from Pakistan, within months they had another child with exactly the same condition."
Research for BBC2's Newsnight in November 2005 showed British Pakistanis accounted for 3.4% of all births but have 30% of all British children with "recessive disorders".