Mr Mugabe and Mr Tsvangirai shook hands at their first meeting in a decade
Zimbabweans have warmly welcomed a deal setting a framework for talks on the country's political crisis.
Residents in Harare and Bulawayo told the BBC they were excited at news of the agreement, saying they hoped it would allow a return to normal life.
The deal says power-sharing talks between President Robert Mugabe's Zanu-PF and the opposition MDC should be completed within two weeks.
South African officials said the talks would begin immediately.
Meanwhile, EU officials said they were extending sanctions against allies of Mr Mugabe, adding 37 names to the list of 131 people subject to a travel ban and assets freeze.
The signing ceremony, which MDC leader Morgan Tsvangirai described as "historic", was the first time he had met Mr Mugabe in a decade.
On Tuesday, he said in a statement that the deal "offers the most tangible opportunity in the past 10 years to improve the lives of our fellow citizens".
"But our signatures alone do not guarantee that we will be able to make the most of this opportunity," he warned.
The talks, to be held in South Africa, are expected to focus on a possible power-sharing agreement, how to revive Zimbabwe's devastated economy and ending the political violence.
But the BBC's Jonah Fisher in Johannesburg says the deal has not settled any of key issues, such as how two parties work together and - crucially - what happens to Mr Mugabe.
Mr Mugabe and Mr Tsvangirai have been locked in a bitter dispute over this year's presidential election.
Mr Tsvangirai, who leads the opposition Movement for Democratic Change (MDC), won the first round of presidential elections at the end of March, but official results gave him less than the 50% needed for outright victory.
Mr Mugabe won the second round after Mr Tsvangirai withdrew, complaining of a campaign of violence against his supporters.
The poll was widely criticised by Western powers and by a small group of African countries.
Zimbabweans expressed hope that Monday's deal would bring an end to the political unrest.
One resident in the capital, Harare, said Zimbabweans were "looking forward to peace and development".
"We should make sure people have enough food and enough medicines in hospital," said another.
In Bulawayo, Zimbabwe's second city, journalist Themba Nkosi said both government and opposition supporters were desperate for a return to normal life, wanting schools that had closed during the post-election violence to reopen.
Zimbabwean exiles in South Africa were more sceptical.
Solomon "Sox" Chikohwero, Vice-Chairperson of the Zimbabwe Diaspora Forum, told the BBC there would only be cause to celebrate once a power-sharing deal was signed.
South African presidential spokesman Mukoni Ratshitanga said the talks would begin at an undisclosed location in Pretoria on Tuesday afternoon.
However, MDC spokesman Nelson Chamisa said the party's lead negotiator, Secretary-General Tendai Biti, remained in Harare.
The MDC says at least 120 of its supporters have been killed, about 5,000 abducted and 200,000 forced from their homes since the first round of the elections, in a campaign of violence by pro-Mugabe militias and the army.
Cabinet ministers and military officials have denied the charges.