Mikhail Babich announced he was leaving the job only three months after arriving in Chechnya from central Russia to run the pro-Moscow government.
A public dispute erupted in January between Mr Babich and the head of the administration, Akhmed Kadyrov, after Mr Kadyrov appointed a local finance minister without consulting the prime minister.
The BBC's Moscow correspondent, Steve Rosenberg, says the fact that Moscow has not come out in support of Mr Babich, seen very much as the federal government's man on the ground, shows how dependent it is on Mr Kadyrov, an ethnic Chechen and former rebel leader, to run the region.
On Saturday, Mr Babich would only say he was leaving his post to take up a new position, without giving details.
Mr Kadyrov is now due to meet his ministers and President Vladimir Putin's representative in the wider, south Russian region, Viktor Kazantsev, to appoint a replacement by Monday.
Our correspondent says the episode is embarrassing for the Kremlin, which needs peace and stability in Chechnya ahead of the referendum on a new Chechen constitution due to be held in March.
Financial review
When Mr Babich,33, was appointed in November, he was seen as the eyes and ears of Moscow, tasked with keeping control of the region's finances.
When Mr Kadyrov replaced the existing Finance Minister, Sergei Abramov, with his own candidate, Eli Isayev, Mr Babich cried foul, saying the move was unconstitutional.
Under Russian law, only the Chechen regional prime minister may appoint cabinet ministers.
Mr Babich suggested that the replacement was connected to an upcoming review of Chechnya's finances.
Mr Kadyrov countered that Mr Babich's remarks were "based on a distortion of facts".
Our correspondent adds that Mr Kadyrov will be hoping that support from Moscow will help his chances in a possible presidential election in Chechnya in 2004.
In other developments related to Chechnya: