Mr Field said it is a 'tragedy' that Mr Brown is not enjoying being PM
Prime Minister Gordon Brown is confident he will still be the leader of the Labour Party at the next general election, Downing Street has insisted.
No 10 spoke out after ex-minister Frank Field said he would be "very surprised" if Mr Brown was still in charge then.
Mr Field said Mr Brown was prone to rages and "unhappy", adding that the PM should talk to his loved ones and "see what they say and act on their advice".
But the spokesman said Mr Brown would not be distracted from the big issues.
The comments from Mr Field follow memoirs by Cherie Blair, John Prescott and Lord Levy about Mr Brown's relationship with Tony Blair.
Mr Field's assessment of the prime minister's position, along with the memoirs and poor local election results came as Mr Brown seeks to regain the political initiative.
Mr Brown is unveiling a round of longer term policies, including one to reform social care for Britain's ageing population, in a bid to turn his fortunes around.
Downing Street confirmed that Mr Brown would publish his draft Queen's Speech on Wednesday.
However, in a damning critique, Mr Field told the BBC's World Service it was obvious to everyone that Mr Brown was "unhappy" in the job.
"That's clearly part of the tragedy - on a personal level as well for a party, government and country level - that somebody whose real aim in life is to be prime minister, now has the task and seems so lacking in enjoyment in trying to carry it out," he said.
Pressed on whether he could see Mr Brown leading the party into the next general election, the former welfare minister said: "I would be very surprised if he's still leader of the Labour Party then and therefore, leading us into the election campaign."
He said the abolition of the 10p tax rate could be the trigger to bring Mr Brown down.
"If he asked my advice, I would say to him, 'talk to the people that you most love and who most love you and see what they say and act on their advice'," he said.
"Certainly with this Budget coming up, with the 10p cut in the first rate of tax, which has caused anger not of the like I have seen on the backbenches ever before.
"If we don't get a satisfactory deal, I think there are enough members on the Labour backbenches who will, with others, block the Budget going through and that would make his position intolerable."
In his memoirs, Mr Prescott said Mr Brown was prickly, annoying and could "go off like a volcano".
Mr Field said he had also been on the wrong end of Mr Brown's "tempers of an indescribable nature".
The prime minister's official spokesman would not be drawn on those claims.
He said: "The prime minister's view is that he's not going to be distracted by this kind of stuff and what he is focusing on are the big issues facing the country and are of concern to the people in the country."
It isn't news that Frank Field is not a member of the Gordon Brown fan club
Pressed on whether Mr Brown was confident he would lead the party at the next election, the spokesman replied: "Of course he is."
He said he did not know if Mr Brown planned to write his own memoirs or kept a diary.
Health Secretary Alan Johnson said he was not familiar with Mr Field's view of the prime minister, and dismissed the claims as the "knives coming out".
He said Mr Field had only been a minister for one year, whereas he had worked with Mr Brown for nine years.
"I have never been shouted at once - I've never had the experiences that Frank describes," he told BBC Radio 4's Today programme.
"I'm not a great Brown fan club leader, but I respect him as a really, really good, decent, able politician. Is he perfect? No he's not, nor is anyone else in the world.
"Can't we just set all this true confessions stuff to one side and talk about things that are really important to people?
"Why do I think it's all coming out now? Because people's biographies are being published now and because some people just see an opportunity to put the knife into somebody they dislike."
Mr Johnson said that all prime ministers - including Tony Blair, Sir Winston Churchill and Baroness Thatcher - had people who did not like them.
"Frank Field doesn't get on with Gordon Brown - well we've known that for 10 years," he added.
Conservative leader David Cameron said the government was "beginning to resemble a sort of bizarre soap opera, where they seem more concerned with settling scores with each other than actually running the country.
"People will actually ask: 'How can I possibly get leadership from this government on the things that matter, like the cost of living, when they're so busy fighting themselves?'" he said.
"That's why people are increasingly looking to the Conservative Party and why I'm so determined that we're really going to win over their trust."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.