At least two suspects managed to flee after the incident, which occurred in the port city of al-Mukalla, about 850 km (530 miles) south east of the capital, Sanaa.
They were named as Shaker Hamel and Abdel Hakem Abduo, wanted for the October bombing of a French tanker in the Gulf of Aden which killed a Bulgarian crew member, the Associated Press news agency reported.
Yemeni police officers had surrounded a three-storey building in the city after being tipped off that al-Qaeda members were living there, an official said.
Shots were exchanged and at least six officers were wounded, with two later dying in hospital of their injuries.
Previous attacks
More than 20 suspects have already been arrested over the Limburg attack in the Gulf of Aden, in which a small boat loaded with explosives rammed the tanker, blasting a hole in its side.
Yemeni news reports said that a French security team recently returned to the country to examine the government's investigation into the blast.
The incident echoed a similar attack on US warship the USS Cole in the port of Aden in 2000 which left 17 US sailors dead.
Both attacks have been blamed on Osama Bin Laden's al-Qaeda network, which the US also accuses of the 11 September attacks.
In November the US Central Intelligence Agency killed six al-Qaeda suspects in a car explosion in Yemen.
~RS~a~RS~International~RS~q~RS~~RS~z~RS~02~RS~)