The resignation has been described as too little, too late and a recognition that India's security apparatus failed, the BBC's Adam Mynott reports from Mumbai.
The attacks have also increased tensions with Pakistan after allegations that the gunmen had Pakistani links.
Islamabad denies any involvement, but India's Deputy Home Minister Shakeel Ahmad told the BBC it was "very clearly established" that all the attackers had been from Pakistan.
Indian troops killed the last of the gunmen at the Taj Mahal Palace hotel on Saturday.
Anger in Mumbai
Prime Minister Singh was speaking at the cross-party meeting in Delhi.
On the agenda are the creation of the new agency and new anti-terror laws.
Indian prime minister on anti-terror plans
Mr Singh was quoted by Reuters news agency as saying he planned to increase the size and strength of the country's anti-terrorist forces.
As few as 10 militants may have been involved in Wednesday's assault which saw attacks in multiple locations including two hotels, a major railway station, a hospital and a Jewish centre.
While the vast majority of victims were Indians, at least 22 foreigners are known to have died, including victims from Israel, the US, Germany, Japan, Canada, Australia, Italy, Singapore, Thailand and France. One Briton, Andreas Liveras, was also killed.
When coastguards boarded the vessel, they found... a satellite phone and GPS tracker that possibly belonged to the trawler's crew.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.