It was not known where the missile had been fired from, there had been no warning.
There has so far been no admission of responsibility for the attack but the involvement of dissident Irish republicans has not been ruled out.
Staff at underground stations felt tremors and drivers felt their cars shake under the force of the blast, just before 2200 BST on Wednesday.
Deputy Assistant Commissioner Alan Fry, head of Scotland Yard's Anti-Terrorist Branch, said the missile had caused "minimal damage".
It had not disrupted the work of the Secret Intelligence Service (SIS), he added.
It was not known where the missile had been fired from, there had been no warning.
There are no reports of any casualties.
Mr Fry said he was "keeping in mind" dissident Irish republican groups, but was not ruling out any other possible perpetrators.
"Examination of the exterior of the building would show that some form of small missile has hit the exterior at about the eighth floor leading onto the Albert Embankment," he said.
"At this moment in time you will appreciate it is far too early for me to speculate.
He said it was "impossible to judge" why the building had been targeted.
But he said: "The sheer nature of the work of the Secret Intelligence Service must mean that there are other terrorist groups throughout the world who might see their premises as a potential target."
Mr Fry referred to attacks which had caused disruption in London during the summer.
In July, the Real IRA was understood to have been responsible for a bomb in Ealing, west London, and in June, Hammersmith Bridge was closed after a device was planted on one of the supports.
"As I have warned, we all need to be vigilant," Mr Fry said.
"We have a genuine threat of terrorism in London and that is against a number of targets, as we have seen.
"We have seen the attack on Hammersmith Bridge, we have seen the attack on the railway system and tonight we have seen an attack on the Secret Intelligence Service headquarters."
Police would carry out a "painstaking forensic examination" of the scene and the surrounding area.
Mr Fry said he doubted the missile was a mortar, which he would have expected to cause more damage.
He appealed for anyone who had any information, or who had witnessed anything suspicious at the time of the attack, to come forward.
An area of about three-quarters of a mile around the site at Vauxhall Cross, on the south side of the River Thames, has been cordoned off.
Waterloo railway station was closed and is not expected to re-open until at least 0830 BST on Thursday, with no overland commuter, intercity or Eurostar trains running into the station.
Click here for the latest BBC London travel news
Vauxhall Bridge and roads around Vauxhall Cross will remain closed for much of Thursday.
A Foreign Office spokesman said the Foreign Secretary, Robin Cook, was being "kept closely in touch with developments", and was "relieved that there have been no casualties".
Earlier this year, both MI6 and the domestic intelligence service were warned that their ability to gather secret intelligence could be compromised by recent security lapses.
Anyone with any information about the attack should contact the Anti-Terrorism Branch on 0800 789321.