Mr Mousavi had called supporters to take to the streets wearing black in memory of those shot by members of the pro-government Basij volunteer militia on Monday.
The protesters heeded the call, waving black banners and holding aloft placards asking, ''Why did you kill our brothers?'' Some banners carried pictures of the dead.
The opposition leader attended the rally, wearing a black shirt and suit. He addressed the crowds through a loudspeaker, with loud chants of support breaking the general silence.
Press TV, the English-language version of Iranian state television, reported that the address was brief and that Mr Mousavi called for calm and restraint.
A BBC correspondent at Thursday's Tehran rally
The crowd is enormous. It is impossible to put any kind of estimate on the numbers because it is so spread out. It starts down in a large square, Iman Khomeini square, and is filling the streets and avenues all around.
The most you can say is that there are several hundred thousand people here. And just like Mr Mousavi asked, the great majority are wearing sombre clothes. Many are in black, especially the women, who have turned out wearing black chadors.
It is, in effect, a funeral procession for the eight people who were killed here in Tehran on Monday, and the perhaps seven or so who have been killed in other parts of the country. But the most remarkable thing about this demonstration is the complete silence. The only sound is a certain amount of conversation. There is no shouting, no chanting - just a really dignified silence.
Reporter Marie Colvin of the UK's Sunday Times newspaper told the BBC from Tehran that both sides seemed to be trying to avoid major clashes but the situation was "a tinder-box, very edgy, so it's very hard to predict".
The day of mourning was also observed outside Tehran.
One protester, Ali, took part in a silent sit-in at a shrine in Shiraz, south-western Iran, to remember those killed.
He told the BBC: "There are about two or three thousand people here, all sitting in silence in the big courtyard inside the shrine. Police won't do anything because we are in a holy site."
Heavy restrictions have been placed on the BBC and other foreign news organisations.
Reporters are not allowed to cover unauthorised gatherings or move around freely in Tehran - but there are no controls over what they can write or say.
President Ahmadinejad has meanwhile appeared on state television qualifying his earlier comments that compared the protesters to angry football fans.
"I meant those who riot, those who set fire [to buildings] and attack people... I said those individuals... are aliens to our nation."
He added: "I stressed that this election victory belonged to 70m Iranians and the 40m who took part in voting. Everyone is a winner. Iranians are very much valued and respected, and the cabinet belongs to all Iranians".
The protests came as the powerful Guardian Council said it had invited Mr Mousavi and fellow defeated candidates Mehdi Karroubi and Mohsen Rezai to a meeting on Saturday to discuss their election complaints. It is not known if the three candidates have accepted the invitation.
Guardian Council spokesman Abbasali Khadkhodai said a "careful examination" of the 646 complaints from the three candidates had begun.
Mobile phone footage of Mousavi at the rally in Tehran on Thursday
The council earlier this week said it would carry out a partial recount, but had ruled out a re-run of the poll demanded by Mr Mousavi.
On Friday, Iranians will be listening closely to the address of Supreme Leader Ayatollah Ali Khamenei, who is due to deliver the sermon at Friday prayers.
In other events on Thursday:
• A counter-rally was held outside the prosecutor's office in Tehran in which hard-line students protested against the role of former President Akbar Hashemi Rafsanjani and his family. The Fars news agency said his daughter, Faezeh, who addressed an opposition rally on Tuesday, and her brother Mehdi had been barred from leaving Iran over their alleged role in the unrest.
• Ebrahim Yazdi, a foreign minister after the 1979 revolution and now leader of the Freedom Movement of Iran, was arrested while undergoing tests at a hospital in Tehran, a spokesman for his organisation said.
• The Assembly of Experts - Iran's top clerical body responsible for appointing the supreme leader - welcomed the election turnout but made no mention of the result. It is headed by Mr Rafsanjani.
• Iranian Nobel Peace Prize winner Shirin Ebadi called for the election result to be annulled, Reuters news agency reported.
Are you in Iran? What do you think of the current situation? If you have any information you would like to share with the BBC you can do so using the form below:
Send your pictures and video to firstname.lastname@example.org or text them to +44 7725 100 100. If you have a large file you can upload here.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.