The former shoeshine boy has played on his rags-to-riches story.
Mr Toledo's rival, former president Alan Garcia, congratulated him as partial election results indicated that he had secured 52% of the vote.
But Japanese Prime Minister Junichiro Koizumi said that Tokyo would not bow to pressure from Peru to extradite Mr Fujimori.
Mr Toledo has pledged to secure the return of the former president to face questioning over his role in the corruption scandal.
The BBC correspondent in Lima, Peter Greste, says the centre-left economist has widely been seen as the best man to drag Peru out of its current economic crisis.
He trained at Harvard and worked for the World Bank before returning to Peru to enter politics.
Indian roots
As well as calming the financial markets, his election will appeal to the nation's indigenous people. Mr Toledo was one of 16 children born to a poor Indian family in the Andes
The former shoeshine boy has played on his rags-to-riches story.
Mr Toledo hailed his victory as a triumph of democracy, decrying the "dictatorship of the past."
Speaking to thousands of supporters celebrating his victory, the 55-year-old pledged that he would never defraud the nation.
"Tonight we share a dream of a Peru that is more just, that has no corruption, a Peru that has justice and equality for all," he told them, as fireworks lit up the sky.
Disillusioned
With 89% of the votes counted, Mr Toledo had over 52% of the votes compared with under 48% for Mr Garcia.
The election - vital for a country disillusioned by political corruption and scandal - appears to have been the most transparent for years.
"The time has come to extend Dr Toledo my congratulations for his triumph on this democratic day," said Mr Garcia, who served as the country's president in the 1980s.
Thousands of ecstatic supporters cheered their new president, who faced them on the balcony of a Lima hotel.
"Brothers and sisters. Tonight is the start of the future," Mr Toledo said to cries of "Long live the president" and his campaign nickname 'Pachacutec' in honour of an Inca emperor.
Personal attacks
The election campaign was bitter.
In a live television debate, Mr Toledo accused Mr Garcia of having a terrible human rights record during his presidency.
In return, Mr Garcia brought up allegations of Mr Toledo's cocaine-taking past and accused him of fathering a child out of wedlock.
Political analyst Mirko Lauer wrote in Peru's La Republica newspaper: "For this society, a clean election in itself is more important than the presidential result."
But the head of the Organisation of American States' mission, Eduardo Stein, said that whoever won would have a rocky six months trying to persuade the Peruvian people to once again trust their politicians.