Georgian President Mikhail Saakashvili has said he is taking charge of the government following the death of his prime minister on Wednesday.
Mr Zhvania played a prominent role in the Rose Revolution
Zurab Zhvania, 41, was found dead in an apartment in Tbilisi, apparently poisoned by gas from a faulty heater.
Police have launched an inquiry, but the interior minister said no foul play was suspected and described the incident as a tragic accident.
"I am taking responsibility for leading the executive," Mr Saakashvili said.
"I am ordering the government to return to work," he added.
Mr Saakashvili has seven days to nominate a new prime minister to parliament.
Mr Zhvania was once an ally of ex-President Eduard Shevardnadze.
But he turned against him and played a prominent role in the 2003 Rose Revolution that ousted Mr Shevardnadze.
Officials say security guards found the prime minister's body after breaking into a flat owned by friends early on Thursday.
Mr Zhvania had gone to the Tbilisi flat of another Georgian official, Raul Usupov, at about midnight local time (2100 GMT) on Wednesday.
Security guards became suspicious when the prime minister failed to answer his phone for several hours.
They broke in at around 0400 to find Mr Zhvania dead in an armchair and Mr Usupov dead in the kitchen.
Giorgi Janashia, the Georgian state prosecutor, said technical experts reported that an Iranian-made gas heater in the flat had not been installed properly and there was no ventilation "which may have caused the fatal gas leak".
He told Georgian radio that blood tests had been sent for analysis, but the bodies of the men showed signs of carbon monoxide poisoning.
But one Georgian MP, Amiran Shalamberidze, has raised suspicions about the death.
He said the accident followed days after a car bomb in Gori, near the border with South Ossetia - one of the breakaway regions with which Mr Zhvania had been trying to negotiate a deal.
"There is the impression that that these tragic facts didn't occur by chance but were the results of interference from the side of certain outside forces," Mr Shalamberidze was quoted as saying by the Itar-Tass news agency.
Russia's Foreign Minister Sergey Lavrov condemned the comments.
Mr Saakashvili held an emergency government meeting on Thursday, which was shown live on Georgian television.
"Georgia has lost a great patriot whose entire life was devoted to tireless and selfless service to our country. I have lost my closest friend, most trusted adviser and greatest ally," Mr Saakashvili said, addressing his ministers.
"I hope you will stand firm, Georgia will stand firm, and I hope I will stand firm because it is firmness that we need the most at present."
Mr Saakashvili said his thoughts were with "Zurab's wife Nino, his mother Irma and his three most beautiful children".
Thousands of people are expected to turn out for Mr Zhvania's funeral which is due to be held on Sunday.
Another Rose Revolution leader, parliamentary speaker Nino Burjanadze, cut short her visit to Italy and is returning to Georgia.