Police have arrested 15 people at ports or airports as they tried to travel to Germany for the World Cup as part of a crackdown on potential hooligans.
It is hoped the police operation will stop troublemakers travelling
Officers used powers in the Football Disorder Act (2000), which allows the arrest of possible troublemakers who have not been given banning orders.
Up to 200 fans with banning orders did not give up their passports before last week's deadline, the Home Office said.
If arrested, they could be jailed for six months and fined £5,000.
About 3,500 people have been subjected to banning orders.
The details of those who should have handed in their passports have been passed to port authorities in the UK and Europe, according to the Home Office.
At least one known hooligan has evaded the security operation after travelling in defiance of a banning order.
He reached the German border before he was caught.
Meanwhile, Cumbria Police said 22 fans in the county subject to banning orders had surrendered their passports ahead of the World Cup in Germany.
The remaining three have now been charged with breaching their banning orders and will appear in court.
Officers checked tickets and travel plans across England, to try to stop anyone flying while on a football banning order.
A man was detained at Stansted Airport on Tuesday and has since been banned from travelling to Frankfurt.
Robbie Varle, 20, from Dunstable in Bedfordshire, was barred from attending any football matches for the next three years under the Football Spectators Act.
Bob Broadhurst, Gold Commander in London for the duration of the tournament, said a few troublemakers would inevitably get through, but many others would be stopped.
"If you keep out those people that follow the hard core then the hard core are isolated," he said.
"If they have not got anyone to fight with then they will not fight.
"They have to think carefully, if they are on the periphery, as if they throw bottles then they may end up with a banning order and not be seeing football for a very long time."
However, Malcolm Clarke, chairman of the Football Supporters' Federation, said he was concerned about the tough use of laws to combat soccer violence.
"I would compare it to the issues surrounding anti-terrorism legislation. There is real danger that in attempting to prevent any trouble at football grounds, people's civil liberties are being affected."