Lord Falconer has denied a report the courts system is failing to cope with the increasing numbers of terror cases.
Terror cases could be tried at high-security court centres out of London
The Lord Chancellor told BBC News it was "possible" other cases could be delayed and more high-security courts were needed, "but there is no backlog".
Earlier, the Crown Prosecution Service revealed a total of 99 defendants were awaiting trial in 34 terror cases.
Counter-terrorism head Sue Hemming told the Times she was not sure the system was "geared up" to deal with them.
She also revealed work had begun to find other court centres outside London where terror cases could be tried.
"There is work at the moment to look at improving court centres and increase the number of courts that can deal with terrorist-related cases and high-security cases generally," Miss Hemming told the newspaper.
Lord Falconer told BBC News terrorism cases were a "priority" and there would always be a court and judge available when they were ready to come to trial.
"There are more terrorist trials than there have ever been before and we need to gear up to ensure that we can deal with them," he added.
"These cases take a significant time to prepare, both by the prosecution and by the defence."
A Crown Prosecution Service spokeswoman said the number of terror cases was "challenging".
But she added: "The agencies are committed to working to meet that challenge while looking at improvements for using regional court centres to deal with the cases."
The government's independent reviewer of the anti-terrorism legislation, Lord Carlile of Berriew, said the "burden... can be shared out among the significant court centres in this country".
"As long as they are all not left to wait in a bus queue in London, I think that they will be dealt with reasonably expeditiously," he added.
Shadow attorney general Dominic Grieve told BBC News "a group of judges who actually specialise in this work" was needed.
"Judges, once they build up an expertise in a particular field and start to specialise in certain trial areas, can usually move the process along faster by having pre-trial hearings, by trying to narrow down the issues that will be dealt with at the trial and by putting pressure on both sides - both the prosecution and the defence - to speed up the process, when there is often a lot of inertia," he said.