Mr Mugabe's speech was occasionally drowned out as they sang that his party was "rotten".
Correspondents say he looked annoyed and raced through the final lines of his speech.
The BBC's Karen Allen says this was humiliating for Mr Mugabe, as the speech was broadcast live on national television.
Meanwhile, the AP news agency reports that an MDC MP was arrested at his home on Tuesday. One of his colleagues is in custody after being detained when he turned up at parliament to be sworn in on Monday.
A second MDC MP was briefly detained on Monday.
The MDC says this is part of a campaign of intimidation but the police say those accused are wanted in connection with murder, rape and political violence.
No senior Zanu-PF officials have been arrested over the violence.
Mr Mugabe arrived at the opening of parliament in the capital Harare to cheers from his supporters outside, the AFP news agency reports.
As he arrived outside the parliament building in a vintage Rolls Royce, many in the crowd chanted: "He is our father. He is our leader."
The MDC MPs were overjoyed at Lovemore Moyo's victory
MDC spokesman Nelson Chamisa told Reuters news agency it was wrong for Mr Mugabe to address parliament before a power-sharing deal was reached.
"The dialogue has not been completed. It is arrogant and reckless for anybody to come and address parliament," he said.
The talks, mediated by South Africa, have stalled over how powers should be divided between Mr Mugabe and Mr Tsvangirai, who would be named to fill the new post of prime minister.
Mr Tsvangirai has opposed the recall of parliament, saying it could jeopardise the talks.
But his protests were ignored.
The new speaker of parliament told the BBC's Network Africa programme that he was excited by his election but also "humbled and honoured" by the show of support from his colleagues.
They broke into song and dance when he was announced the winner of the secret vote.
"I feel this is the beginning of the new era in parliament, the era probably when the executive has to find ways of negotiating with the legislature in order to put through programs," he said.
Mr Moyo said he would remain neutral as a speaker but that his decision-making would be influenced by MDC policies.
Mr Moyo's position means that he will be able to take charge of controversial debates if no power-sharing deal is reached.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.