Iranian students have thrown firecrackers and bricks into the British embassy compound during a protest in the capital Tehran.
About 200 people demonstrated against what they say was the illegal entry into Iranian waters of 15 UK Royal Navy personnel captured nine days ago.
BBC correspondent Frances Harrison said the students were hardliners who were urging Iran not to compromise.
UK Defence Secretary Des Browne said the countries were in direct contact.
The BBC's correspondent, who was outside the embassy when the missiles were thrown, said the demonstrators were hardline Islamist students from Tehran University.
The protesters, whose demonstration was allowed by the Iranian government, were chanting "death to England" and calling for the deportation of the British ambassador, she said.
UK VERSION OF EVENTS
1 Crew boards merchant ship 1.7NM inside Iraqi waters
2 HMS Cornwall was south-east of this, and inside Iraqi waters
3 Iran tells UK that merchant ship was at a different point, still within Iraqi waters
4 After UK points this out, Iran provides alternative position, now within Iranian waters
"They were making speeches, they were calling for the British sailors to be put on trial, for the British government to apologise," she said.
Riot police fired pepper spray into the crowd to prevent protesters surging forward and scaffolding had been erected against the front of the embassy to prevent people trying to get inside.
However, because of the New Year holiday period in the country, many Iranians were not fully aware of the crisis over the captured personnel, she said.
The protest was a "worrying sign" that negotiations over the release of the captured personnel may take time, she added.
The Foreign Office said there had been no damage and no-one had been hurt in the incident.
Britain denies Iran's claims that the UK crew was in its waters when seized on 23 March and is demanding their "immediate" return.
Mr Browne, who is in neighbouring Afghanistan, said both countries were in direct bilateral contact but said the British position had been made clear.
"There is no reason to continue to keep them there. We are anxious that this matter be resolved as quickly as possible and that it be resolved by diplomatic means and we are bending every single effort to that."
IRANIAN VERSION OF EVENTS
1 Royal Navy crew stray 0.5km inside Iranian waters
2 Iran gives set of co-ordinates to back up their claims
3 According to seized GPS equipment, the Royal Navy crew had previously entered Iranian waters at several other points
4 Iran informs Britain of the position where the crew were seized, inside Iranian waters
The support from the international community should make it clear to Iran "that their responsibility is to release our detained personnel", he added.
President George W Bush earlier condemned Iran's "inexcusable behaviour" after the capture of the UK personnel and said he would "strongly support" the British government over the crisis.
His calls were echoed by German Chancellor Angela Merkel who said Britain had the "full solidarity of the European Union".
And the Muslim Council of Britain has written to the Iranian ambassador in London, Rasoul Movahedian, to appeal for the release of the UK personnel.
"The most important priority now must be to see these service personnel released without delay so that they can return to their families," said secretary general Muhammad Abdul Bari.
Meanwhile, Iranian Foreign Minister Manouchehr Mottaki said Iran was waiting for a "change in attitude" from the UK and a "moderate approach" to its requests.
He confirmed the receipt of a note from the British embassy about the military personnel, Iranian television reported.
"This note contains many points which will be considered," he said. "However, we are waiting for a change in the attitude of the British and a moderate approach by this country towards Iran's legal requests."
Former hostage Terry Waite, who was held captive for 1,760 days in Beirut before being released in November 1991, has offered to travel to Iran to negotiate with those holding the Britons.
Mr Waite said threatening the Iranian government was counterproductive and said he would be able to "cut through some of the rhetoric".
The Britons, based on HMS Cornwall, were seized by Revolutionary Guards as they returned from searching a vessel in the northern Gulf.