Downing Street has sought to distance Gordon Brown from one of his minister's concerns about first cousins marrying.
No. 10 said the issue was best dealt with by healthcare experts.
Environment minister Phil Woolas said on Sunday there was a raised risk of children having genetic problems if their parents were cousins.
He said: "If you are supportive of the Asian community then you have a duty to raise this issue."
No 10 said Mr Woolas was speaking as a local MP and such matters were best dealt with locally by health officials.
Mr Brown's spokesman said: "The government's position is we believe these matters are best addressed locally, by local members of the community as well as by professional healthcare advisers."
Downing Street's comments came after a Labour MP said she welcomed the debate about first cousins marrying and having children - saying she had been "fretting about this for 10 years".
Ann Cryer, MP for Keighley, says the genetic problems have a "large impact" on her sizeable Pakistani community.
Genetics professor Steve Jones said there was a higher risk but said drinking or smoking in pregnancy was "as bad if not worse".
Professor Jones, from University College London, told BBC Radio 4's Today programme "mortality and disability go up by almost twice in cousin marriages compared with generally".
But he said if it was an "elephant in the room", then it was a "small elephant", saying that in Bradford it involved just five out of 70 infant deaths.
That figure was still too high, he said, and the issue should be raised to educate people of risks as "a matter of public health".
Mr Woolas, an environment minister who represents ethnically-diverse Oldham East and Saddleworth, told the Sunday Times: "If you have a child with your cousin the likelihood is there'll be a genetic problem.
"The issue we need to debate is first cousin marriages, whereby a lot of arranged marriages are with first cousins, and that produces lots of genetic problems in terms of disability [in children]."
Mrs Cryer raised the issue two years ago after research showed British Pakistanis were 13 times more likely to have children with recessive disorders than the general population.
"The vast majority of marriages in the Muslim community of Bradford, 80% are trans-continental and the majority of those are to cousins and many of those do result in either infant mortality or recessive disorders that I have seen," she told Today.
"Certainly in one family they have three daughters - it seems to have been carried in the female gene - all three daughters have had to have liver transplants.
"One girl has had, I think, four liver transplants to the point where she's so worn down by all of this the poor girl is having to receive psychiatric help as well, no-one wants to wish that sort of thing on their children.
"I'm hoping that now we've got this debate going that the leaders of the mosques and the community centres will actually encourage debate and encourage parents preparing for a marriage to move away from cousin marriages."
Mr Woolas stressed the marriages, which are legal in the UK, were a cultural, not a religious, issue and confined mainly to families originating in rural Pakistan.
But he also told the Sunday Times: "If you talk to any primary care worker they will tell you that levels of disability among the... Pakistani population are higher than the general population. And everybody knows it's caused by first cousin marriage.
"Awareness does need to be raised but we are very aware of the sensitivities," he added, pointing out that many of the people involved were the products of such marriages."