US and Iraqi troops have raided houses and buildings in Baghdad's Sadr City, close to where five Britons were kidnapped on Tuesday.
Witnesses said police units sealed off the street outside the ministry
The suburb is a stronghold for the Shia militia, the Mehdi Army.
Iraqi foreign minister Hoshyar Zebari said he believed the group, not al- Qaeda, might be behind the abductions.
The Anglican vicar of Baghdad, Canon Andrew White, said the kidnapping could be linked to the recent killing of a radical Shia cleric by UK troops.
BBC correspondent Jim Muir said the raids in a Shia area were the "clearest physical indication so far" that suspicions were falling on the Shia militia.
The US military said a number of militants had been arrested during the raids but would not confirm any link with the missing Britons.
Foreign Secretary Margaret Beckett said the UK government was working closely with Iraqi authorities and doing all it could to secure the men's release.
She said their families were being offered support at this "distressing time".
The five men - a computer expert and four bodyguards - were taken from the finance ministry building in Baghdad.
The kidnappers wore police uniforms and staged the capture without firing a shot, senior Iraqi officials said.
Witnesses said Palestine Street was sealed off at both ends and the kidnappers walked past ministry guards.
A police source told the BBC that dozens of police vehicles were used in the operation.
Mr Zebari said the kidnappings represented a "very serious challenge... to the government itself".
The kidnappers probably had connections with local police in the area, he told the BBC's Today programme.
"The number of people who were involved in the operation, to seal all the buildings, to set roadblocks, to get into the building with such confidence, [they] must have some connection."
Canon Andrew White said there was "very likely a connection" between the kidnappings and the death of Abu Qadir, also known as Wissam Waili, a leader of the Mehdi Army militia, who was killed in Basra on 25 May.
He said: "The worrying thing is this is obviously not a case for ransom demand; economic hostage-taking is fairly easy to deal with."
UK Prime Minister Tony Blair, speaking during a trip to Libya, said: "We will do everything we possibly can to help."
British embassy officials in Iraq are following up the case and the Iraqi government has set up a special operations room.
Cobra, the UK government's emergency committee, is expected to meet for the second time later on Wednesday.
The four kidnapped security guards were working for Canadian-owned security firm GardaWorld.
The company is one of the biggest suppliers of private security in Iraq, and is mainly staffed by Britons.
Private contractors are known to be unpopular with Iraqis.
Anas Al-Tikriti, an Iraq expert involved in the negotiations to secure the release of Briton Norman Kember, said mercenaries, private contractors and private firms were considered to be in Iraq "filling their boots".
Iraqi police are heavily infiltrated by militias
"As much as the Iraqis despise and loathe the occupiers, they loathe and despise those mercenaries a hundred times more," he said.
The kidnapped computer expert was working for BearingPoint, a US management consultancy which has worked on development projects in Iraq since 2003.
This is thought to be the first time Westerners have been abducted from a government facility.
BBC security correspondent Frank Gardner said the Iraqi government now had a few questions to answer after such an "incredibly brazen raid" in broad daylight.
He said there were suspicions of low or middle-level connivance in the police, known to be heavily infiltrated by Shia militias.
A team of experienced police hostage negotiators had already been assembled, and extra staff had been flown to the British Embassy in Baghdad, he added.
About 200 foreigners of many different nationalities have been kidnapped in Iraq over the past four years, though the number has fallen dramatically since a few years ago.