Senegal hopes it can bring the sides together
The court declared void a previous decision by the country's High Constitutional Court (HCC), the body which accepts or rejects election results.
The initial results, which suggested no candidate won outright, were contested by opposition leader Marc Ravalomanana, who in February declared himself president and set up a rival administration.
The ruling comes as Mr Ravalomanana and incumbent president Didier Ratsiraka are in Senegal, for talks aimed at defusing the increasinglly violent political crisis.
Fraud allegations
The supreme court said it was annulling the results because Mr Ratsiraka had changed six of the HCC's nine members just weeks before the ballot was held.
The BBC's correspondent in Antananarivo says the ruling is considered a victory for Mr Ravalomanana, who had argued that the election was rigged.
Our correspondent says there will now be a recount, but adds that it's unclear whether election officials will be able to find all the original ballot papers.
On Wednesday the two rivals held separate meetings with African leaders in the Senegalese capital, Dakar, to try to resolve the crisis.
Speaking after talks with the presidents of Senegal, Ivory Coast and Benin, and the Secretary General of Organisation of African Unity, Mr Ravalomanana said no agreement had been reached, but the talks were progressing.
Fresh violence
More than 35 people have been killed in a conflict which has left the country with rival governments, two capitals and splits in the armed forces.
In Madagascar's second city, Fianarantsoa, three more people died on Tuesday.
An army general loyal to Mr Ratsiraka was shot dead in hospital where he was being treated for gunshot wounds.
Sources at the hospital said three hooded gunmen burst in and shot Raymond Randrianaivo seven times.
He had previously been wounded in an exchange of gunfire between two groups of Ratsiraka supporters, who mistook each other for supporters of his rival.
Earlier two women were killed by stray bullets.
Fianarantsoa is held by Ravalomanana supporters, but the provincial governor - who backs Mr Ratsiraka - is barricaded in his official residence and protected by armed guards.
At the weekend, five soldiers died and 18 others were injured in a struggle for control of the city.
No breakthroughs
The two leaders have been invited to Dakar on the sidelines of an African development conference.
The Organisation of African Unity's Secretary General Amara Essy is working for a new dialogue, despite the failure of OAU missions to Madagascar in February and March.
Senegal's President Abdoulaye Wade says he and other African leaders are ready to assist, but first have to hear what Mr Ratsiraka and Mr Ravalomanana want.
But Mr Ravalomanana said last month that he would only talk to his rival about how to get the island back on its feet - not about his political position.