Qatada once called on British Muslims to martyr themselves
Islamic preacher Abu Qatada has been granted bail with a highly unusual 22-hour curfew by the Special Immigration Appeals Commission (SIAC).
Qatada last month won an appeal against deportation to Jordan but remained in prison. He could now be free in weeks.
The Home Office, which is appealing against the deportation decision, said it was extremely disappointed.
Qatada, a Palestinian-Jordanian, was convicted in his absence in Jordan of terrorist offences in the 1990s.
Home Secretary Jacqui Smith said: "I am extremely disappointed that the courts have granted Abu Qatada bail, albeit with very strict conditions.
"Public safety is our main priority and we will take all steps necessary to
protect the public.
"I am already seeking to appeal the Court of Appeal's decision that it is not safe to deport Qatada and we will continue with deportation action with this and the other Jordanian cases."
Qatada is being held in Long Lartin jail in Worcestershire, but could be released within weeks.
He became one of the UK's most wanted men in December 2001 when he went on the run on the eve of government moves to introduce new anti-terror laws allowing suspects to be detained without charge or trial.
In October 2002 the authorities tracked him down to a council house in south London and took him to Belmarsh Prison.
He was eventually freed on bail in March 2005, but was made the subject of a control order to limit his movements.
In August that year he was taken back into custody pending the extradition to Jordan.
SIAC had originally dismissed the firebrand preacher's appeal against deportation on the grounds that Amman had signed a "memorandum of understanding" with the UK.
'Prevention and prosecution'
These agreements, sometimes called MOUs, are designed to give reassurance that countries with poor human rights records will not torture or ill-treat anyone returned to their soil.
However, the Court of Appeal concluded SIAC had misdirected itself in law and its decision could not stand because of the issue of evidence obtained by torture in Jordan.
At the time, Home Office minister Tom McNulty said he was confident Qatada would be deported, adding that "in the meantime, he remains behind bars."
Shadow home secretary David Davis said the situation "undermines the government's assurances that memorandums of understanding are the solution to deporting terror suspects".
He said: "The government should at last answer our calls to focus on prevention and prosecution - for example by allowing the use of intercept evidence in court - rather than just trying to deport these individuals once they are here".
The shadow home secretary also called for the creation of a "dedicated UK Border Police" in order to "prevent foreign terror suspects from entering the country in the first place".