Some now fear the army will take revenge
Police shot dead three soldiers on Wednesday in the town of Jolo, bringing the death toll in two days of clashes to at least 21.
On Tuesday, at least 18 people were killed in a gunfight between police and marines during a rally to support the jailed former governor, Nur Misuari.
Mr Misuari's successor as governor of the autonomous Muslim area, Parouk Hussin, told BBCs East Asia Today programme there was an "ongoing investigation" into what happened, but that those responsible would be made accountable.
"Each side is blaming the other for the incident," he said.
'Butchered'
The soldiers who died on Wednesday were travelling in an army jeep close to the public market.
One survivor who was being treated in hospital for shrapnel wounds in his back said his colleagues were "butchered like animals".
"I jumped out of the jeep, and ran and ran and rolled on the street and hid inside the toilet of a house until army reinforcements arrived," said Private Rizaldy Laygan.
"I do not know why we were attacked."
There are unconfirmed reports that a civilian also died in the fighting.
Misuari connection
Mr Misuari, a former Muslim rebel leader, was deported from Malaysia to the Philippines a week ago, to face charges of inciting rebellion.
More than 100 died during the uprising last November.
The police officers involved in the current violence are believed to be former members of Mr Misuari's Muslim separatist group, the Moro National Liberation Front (MNLF), who were integrated into the police after a 1996 peace accord.
The MNLF stands by the peace deal, but following the Misuari-led rebellion, up to 1,000 members have joined renewed fighting with government troops.
The police unit - which includes former bodyguards of Mr Misuari - is being withdrawn to another province.
The provincial governor, Yusop Jikiri, said troops were being withdrawn to barracks to avoid further clashes.
Jolo town is the capital of the southern province of Sulu, where Mr Misuari has his power base.