Early results show that voters massively rejected the country's two traditional parties in favour of independent candidates, many of whom back Alvaro Uribe.
Mr Uribe, known for his tough views on dealing with Colombia's leftist rebels, already has a massive lead in opinion polls.
The Conservative Party and the Liberals have alternated power in Colombia since the 19th century but by Sunday night they were routed.
Figures for the upper house of Congress, the Senate, show that the Conservative Party of President Andres Pastrana has lost four of their 17 seats while the opposition Liberals were down from 56 to 30.
Figures for the lower house, the House of Representatives, are still preliminary, the French news agency AFP reports.
Sensing defeat, the Conservatives' leader, Carlos Hoguin resigned on Sunday night.
"It doesn't make sense to continue being the party's president," he said.
New faces
Scooping up the majority of the seats in the 102-member Senate were the independent candidates who are seen to favour either Mr Uribe or a former leftist rebel, Antonio Navarro Wolf.
A recent opinion poll gave Mr Uribe, a former Liberal, 60% of preferences for the presidential election, with his closest rival at 24%.
The former governor of Antioquia has built his support on calls for a crackdown on the leftist rebel group, the Revolutionary Armed Forces of Colombia (FARC).
He has said he would hold talks with the FARC but rules out any safe haven for them like the one allowed them by President Pastrana until this February when negotiations collapsed.
Mr Navarro, a former leader of the now disbanded M-19 leftist rebel group, won a seat in the Senate himself along with his supporters.
Successful election
Sunday's election passed off relatively peacefully despite threats by FARC, which refused to recognise it, and pressure from the right-wing militant group the United Self-Defense Units of Colombia.
Officials said at least 150,000 police and troops had been deployed to safeguard the election.
Rebel threats restricted the movements of some candidates during the campaign period.
Some were forced to send videos of their speeches to rallies, while at least one was reduced to dropping leaflets from an plane over towns he could not visit.
Five of the candidates for the election are being currently being held hostage by the FARC - the rebels have about 750 hostages, including presidential candidate Ingrid Betancourt, whose name will be on the ballot in the May 26 presidential election.
The FARC hope to use the hostages as a bargaining chip to force the government to free imprisoned members of the rebel group.
First results indicated that none of the five would be re-elected.