Richard Watson talks about Monday's Panorama Muslim First, British Second
Britain is once again searching for new answers to terrorism and radicalisation. We may not have had a major terrorist attack since the London bombings of July 2005 but the ideological battle against al-Qaeda is being lost at home.
Take the case of Nicky Reilly, a young man with Asperger's Syndrome who lived in Plymouth with his mother.
He was persuaded last year that he would join the ranks of the martyrs if he blew himself up in a packed family restaurant in Exeter.
Fortunately his bomb-making skills were poor, but what is worrying the security services is the intent - he had been convinced by as yet unknown hands that he was acting in the name of God.
Nicky Reilly, the gentle giant as he was known, had never stepped abroad but had been infected by al-Qaeda's ideology in Britain.
Recent demonstrations in London against Israeli attacks in Gaza are causing concern.
They have exposed the raw wounds of grievances felt by many Muslims about Britain's stance on Muslim affairs abroad.
Legitimate political dissent was exploited by a minority of violent extremists to bolster their hatred of Britain.
"Let's have a... war", one of them shouted as missiles were thrown at the police. From this pool, new terrorists may come.
Bomb attacker Nicky Reilly was radicalised in the UK
So what action should the government take? They could continue with the existing policy called Preventing Violent Extremism.
This, as the title suggests, has been focused on those promoting violence.
Investigate them, place them under surveillance, prosecute or deport them, cut out the cancer of extremism and the threat will subside.
Well it has not proved as simple as that. Judging by the number of terrorist plots under investigation by MI5 - more than 200 - there is no shortage of young Muslims who are learning to view Britain with hatred.
When the policy was set in 2006 the government was scared of alienating people so it set the bar of what was "unacceptable" very high.
In other words, only those at the far end of the extremist spectrum were to be challenged.
'Lesser of two evils'
The flipside to this meant that those who denounced violence but who promoted intolerance and held offensive, anti-British views were tolerated.
More than this, some radicals were even courted as part of our counter-terrorism strategy. The idea was that so long as they denounced terror, other views would be ignored.
This was seen as the lesser of two evils - backing certain radicals even if they preached intolerance of homosexuals or women's rights was seen as a way of protecting Britain.
The Gaza protests were illustrative of the divided loyalties some have
But this has been a dangerous path and shows little sign of working.
The radicals took much succour from engagement with the state. Advising the government or the police is an impressive calling card. They can claim their deeply conservative views about life in Britain are being endorsed.
This has helped make these views seem legitimate in the eyes of ordinary Muslim citizens and has added to the climate of Islamic conservatism in Britain today.
Take a walk in any city with a large Muslim population and you will see that second and third generation Muslims are far more conservative than their parents.
Ayesha, a young woman I interviewed for my Panorama film Muslim First, British Second, is an example.
She is a medical school graduate who defends those who preach intolerance of homosexuals.
In terms of her faith, she is also more conservative than her liberal parents, covering herself with the niqab against their wishes.
Forced to change
Those driving counter-terrorism policy believe the old policy has failed. As Panorama will reveal, the government is planning a new approach.
There will be much more emphasis on shared British values and those who preach intolerance will be shunned even if their views do not break the law.
And so the Preventing Violent Extremism policy will effectively change to Preventing Extremism.
Until now efforts have been focused on those who advocate violence
This shift will be uncomfortable for the police - they do not police ideas or ideology unless they contravene the law.
But it is right that they should be careful about who they back and who they fund.
Likewise the government will be more open about criticising Islamic radicals who preach against shared democratic values but stay on the right side of the law.
The argument comes down to the use of public money. It certainly makes sense to sit down and talk with radicals, so long as they do not promote violence and are willing to act within the law.
For pragmatic reasons the police and counter-terrorism officers need lines of communication into radical communities.
Britain also has a long tradition of tolerating political dissent. But moderate Muslims argue using taxpayers' funds to support or endorse isolationist views makes little sense and the government is right to move against this now.
But this is a complex situation, the arguments are not black and white.
While cracking down on divisive preachers may make Britain more resilient to terrorism by creating a stronger sense of community cohesion, this is a 10 or 20-year plan.
Grand sociological aspirations may be desirable but in the shorter term the police and MI5 must worry about the next attack.
Given there is little evidence that the appetite for extremism is fading, the government has little choice but to try a new approach.
Panorama: Muslim First, British Second is on BBC One on Monday, 16 February at 2030 GMT.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.