Abdul-Latif Moussa blew himself up, killing a Hamas policemen sent to arrest him.
Six Hamas fighters, including a senior commander, and several civilians died. The rest of those killed were from Jund Ansar Allah.
About 120 people were injured, and some were in a critical condition, the BBC's Rushdi Abu Alouf says.
Hamas spokesman Tahir al-Nunu said: "We hold Abdul-Latif Moussa and his followers fully responsible for what happened because of his hasty declaration during Friday prayers of a so-called 'Islamic emirate'."
The Jund Ansar Allah is thought to be inspired by al-Qaeda.
The battle lasted several hours
Mr Nunu said: "Anyone who belongs to this group has to immediately hand himself and his weapons over to the Palestinian police and security forces."
Ehab al-Ghsain, a spokesman for the interior ministry of the Hamas government, said the cleric's followers "wanted anarchy" in Gaza.
"We have said there is no chance of a return to anarchy," he said.
Concern for Hamas
On Friday, Hamas fighters fired rocket-propelled grenades at Ibn-Taymiyah mosque, where at least 100 Jund Ansar Allah supporters were holed up.
These declarations [of an Islamic emirate] are aimed towards incitement against the Gaza Strip
Ismail Haniya, leader of Hamas in Gaza
The entire neighbourhood was sealed off as the shooting continued after dark - in what was one of the most violent incidents in the Hamas-controlled Gaza Strip since an Israeli offensive in December and January.
Abdul-Latif Moussa and his armed supporters had sworn to fight to the death rather than hand over authority of the mosque to Hamas.
During his own Friday sermon, the leader of Hamas in Gaza, Ismail Haniya, dismissed Mr Moussa's comments.
"These declarations [of an Islamic emirate] are aimed towards incitement against the Gaza Strip and an attempt at recruiting an international alliance against the Gaza Strip.
"And we warn those who are behind these Israeli Zionist declarations: the Gaza Strip only contains its people."
Abdel-Latif Moussa was surrounded by armed supporters in the mosque
Jund Ansar Allah gained some prominence two months ago when it staged a failed attack on horseback on a border crossing between Gaza and Israel.
The group is very critical of Hamas, which seized control of Gaza in 2007, accusing the Islamist group of not being Islamist enough.
Hamas has cracked down hard on al-Qaeda-inspired groups in the past, the BBC's Middle East correspondent Katya Adler says.
Hamas is concerned they may attract more extremist members, and has forbidden anyone except what it describes as Hamas security personnel from carrying weapons in Gaza, our correspondent says.
She adds that Jund Ansar Allah is not the first al-Qaeda-inspired group to appear in Gaza and it is unlikely to be the last - which is a worry for Gaza, neighbouring Israel and the wider region.
Are you in the region? What do you think of the crackdown? You can send us your views and experiences using the form below:
A selection of your comments may be published, displaying your name and location unless you state otherwise in the box below.
The BBC may edit your comments and not all emails will be published. Your comments may be published on any BBC media worldwide.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.