Paramilitary forces were called in to deal with the siege
Security forces surrounded the mosque in the Kunzar area, about 40 km north of Srinagar, after the militants refused to give themselves up.
A gunfight then ensued which ended after troops flushed out the remaining militant.
The authorities say they belong to the Hizbul Mujahideen group.
Militant groups are fighting to end Indian control of Kashmir, and have been engaged in a campaign for violence for more than decade.
Search mission
Major General Jamwal, the Indian army commander in the area, told the BBC that the mosque was cleared without any damage to the structure.
Soldiers are conducting search operations in the area to see if any more militants are in the vicinity.
There have been a number of mosque sieges in the past year, causing problems for the Indian authorities because a violent confrontation at a place of worship could inflame local feelings.
However, the authorities have warned militant groups against seeking refuge in mosques, saying they will not hesitate to take action in such a situation.
Three militants were killed by security forces last year in Goigam about 40 km north of Srinagar, where they had also taken refuge inside a Muslim shrine.
Strike called
In another development, most shops and businesses in Srinagar were closed because of a strike to protest at alleged human rights abuses.
The strike was called by the manufacturers federation after a Kashmiri Muslim youth was allegedly killed by police in Delhi after his arrest.
Most streets in Srinagar were reported to be deserted.
Human rights groups have accused Indian forces of carrying out serious abuses including summary executions, disappearances and rape.
The Indian authorities say they investigate all reports of rights violations and punish those found guilty.