Tony Blair has said the wearing of full face veils by Muslim women is a "mark of separation" and made some "outside the community feel uncomfortable".
The prime minister also backed Kirklees Council, which suspended classroom assistant Aishah Azmi for refusing to remove her full face veil at school.
But his intervention was criticised by Ms Azmi's lawyer, as the employment tribunal has yet to give a verdict.
He has threatened proceedings against Mr Blair if there is no retraction.
In his monthly press conference, Mr Blair said there was a need for a wider debate about community integration while allowing people to develop their "distinctive identity".
Mr Blair told reporters at his first news conference since MPs returned from their long summer break that a debate was needed on how the Muslim community integrates with British society.
"Difficult though these issues are, I think they have to be raised and confronted and dealt with," he said.
"And then, there's a second issue, which is about Islam itself and how Islam comes to terms with - and is comfortable with - the modern world."
The debate was already going on in "every village, town and city" in the UK, as people sought a balance between "preserving a distinctive identity and integration."
When asked at the news conference if a Muslim woman wearing a veil could make a contribution to society, he replied: "That's a very difficult question.
"It is a mark of separation and that is why it makes other people from outside the community feel uncomfortable.
"No-one wants to say that people don't have the right to do it. That is to take it too far. But I think we need to confront this issue about how we integrate people properly into our society."
Mr Blair made his comments during his monthly press conference
He also said he "fully supported" the way the authority dealt with Aishah Azmi at Headfield Church of England Junior School, in Dewsbury, by suspending her.
But her lawyer, Mr Whittingham, said his comments had "specifically and directly" interfered with the employment tribunal - which had yet to make a decision, which would inevitably be appealed against by the losing side.
He said Mr Blair's interference was a breach of the ministerial code, which requires ministers to uphold the administration of justice.
"We require the Prime Minister immediately to issue a qualification or a retraction," he wrote.
"If comments are not qualified or withdrawn then Mrs Azmi will consider bringing a complaint under the ministerial code and bringing injunction proceedings against the prime minister".
Full face veils became a matter of political debate two weeks ago when Jack Straw said he asked women to remove them, when they visited his office.
Government minister Phil Woolas has also angered some Muslim groups by calling for 23-year-old Ms Azmi to be sacked.
But her MP, Labour's Shahid Malik said ministers had been right to give their views and it had resulted in helpful debate.