The activities of Sheikh Omar Bakri Muhammad are being monitored
David Blunkett is investigating comments made by the Al Muhajiroun group, but said he was wary of stirring up unrest by taking action against them, unless they break the law.
He has been urged to silence radicals such as cleric Sheikh Omar Bakri Muhammad, the London-based leader of Al Muhajiroun, which has issued a decree calling for the murder of "those who wage war against Allah".
The home secretary told BBC Radio 4's Today programme: "We don't ignore him.
"We monitor very carefully what he is saying, who he is inciting, and work out carefully at what point it would be productive to act so he is prevented from doing so.
"I am monitoring, as are the security services and the Metropolitan Police, the actual words he uses."
Mr Blunkett warned that militants may be seeking to create unrest by encouraging authorities to arrest them, which would publicise their cause.
Muslim fears
He said: "Over the last week we have been trying to ensure we keep the balance and don't end up doing precisely what those who want to stir up hatred and community conflict want us to do.
"At the point when we believe it to be no longer acceptable and when he is literally inciting others to go out and engage in acts which threaten other people's lives, we will take action against him.
"I am listening ... to the security service and Metropolitan Police service to be in a position to act decisively when the mark has been overstepped."
Mr Blunkett said the vast majority of Muslims are appalled at the language used by extremists, fearing it could provoke a backlash against all Muslims.
Yousef Bhailok from the Muslim Council of Britain said: "These are not rational people, so I'm afraid their extreme views are certainly not the views of the Muslim community or indeed any part of our civilised society."