Huge celebrations have erupted in Beirut after the Lebanese government announced its resignation following two weeks of popular protests.
Mr Karami denies any involvement in the attack on Mr Hariri
Tens of thousands of people waved Lebanese flags and demanded that Syria remove its troops from the country.
Prime Minister Omar Karami announced the resignation two weeks after the murder of his predecessor Rafik Hariri.
The US hailed it as an "opportunity" for Lebanon, calling for fair elections free of Syrian influence.
Mr Karami said in his announcement: "I am keen the government will not be a hurdle in front of those who want the good for this country.
"I declare the resignation of the government that I had the honour to head. May God preserve Lebanon."
His announcement came as an opposition-sponsored motion of no-confidence in the government was being debated in parliament.
The statement prompted delight from at least 25,000 protesters estimated to have gathered in Beirut's Martyrs Square despite a ban on demonstrations.
"Karami has fallen, your turn will come, Lahoud, and yours, Bashar," the demonstrators chanted, referring to Lebanese President Emile Lahoud and his Syrian counterpart Bashar al-Assad.
Mr Lahoud accepted the resignation of the government and asked it to continue in a caretaker capacity, a statement said.
The immediate reaction from Syria, which backs the Lebanese government, was non-committal, saying only that it was "an internal affair" for Lebanon.
However, a BBC correspondent in Damascus says the Syrian authorities must be worried the situation in Lebanon is spiralling out of their control and might result in a new government demanding the immediate withdrawal of Syria's estimated 15,000 troops in the country.
Both Mr Karami's government and the Syrian government have been accused of involvement in the 14 February assassination of Mr Hariri - charges they deny.
Earlier, Mr Karami - who took office after Mr Hariri resigned last year - said those who accused his government of involvement in the killing "committed a grave injustice".
Before the parliamentary debate opened, MPs observed a minute's silence in memory of Mr Hariri.
A former minister said the government bore partial responsibility for the killing.
"I accuse this government of incitement, negligence and shortcomings at the least, and of covering up its planning at the most... if not executing," the attack, former minister Marwan Hamadeh said.
Protesters watched events from giant TV screens in the square. Many had spent the night there, wrapped in blankets or under tents, before the ban came into force at 0500 (0300 GMT).
Despite army checkpoints, people managed to get to the square and there were no serious clashes.
Many schools and businesses remained shut across the country, following a call by the opposition for a general strike.
Earlier, US Deputy Assistant Secretary of State David Satterfield, on a visit to Lebanon, called on Syria to withdraw its troops in compliance with UN resolution 1559, passed in September.
Syria says not even the Lebanese want a full Syrian withdrawal, but last week it said it would draw troops back from western Lebanon to areas nearer the Syrian border.