Mr Hutton said he thought fellow Blairite minister James Purnell had made "the wrong decision".
"I'm standing down from the cabinet today because I'm leaving frontline politics," Mr Hutton told the BBC.
"I'm not going to be contesting my seat in the next general election and I think it's absolutely right that Gordon, who I'm supporting as our prime minister and party leader, should have a cabinet that's going to take him through the next election and beyond."
He denied that as the fourth cabinet minister to quit in recent days he was "leaving a sinking ship".
Alan Johnson, touted by some backbenchers as a possible leadership challenger, said he backed Mr Brown "to the hilt" to continue as prime minister.
He said he would "never say never" to becoming prime minister at some point, but insisted he could see no circumstances at present where he would mount a bid for the job.
He insisted that Mr Brown was "absolutely the best person for this job" and took a swipe at Mr Purnell, saying: "It is a difficult job at the best of times and it is not a job that his own colleagues should be making more difficult through their own actions."
BBC Political Editor Nick Robinson said Mr Hutton's decision not to attack Mr Brown - coupled with Mr Johnson's loyalty - had shored up his position as prime minister.
But Gordon Brown was not getting the reshuffle he had planned a week ago, he added.
Alistair Darling had turned down a move to the Home Office and Mr Purnell had been "sounded out" about the job of education secretary, which would have paved the way for Mr Brown's ally Ed Balls to become chancellor, but that was not now going to happen.
Mr Balls is expected to remain as schools secretary, sources suggest.
Many Labour backbenchers who were ready to call for a change of leader will now be asking themselves: 'If they're not willing to act to end this, why should I?'
And Labour is still bracing itself for further bad results after heavy losses declared so far in English local elections.
According to the BBC's projected share of the national vote at a general election, based on the results in so far, the Conservatives would poll 38%, the Lib Dems 28% and Labour would be third on 23%.
Cabinet ministers have lined up to back Mr Brown and criticise Mr Purnell's surprise call for Mr Brown to quit, with none so far indicating they were ready to follow his lead.
Foreign Secretary David Miliband, seen as a political ally of Mr Purnell, said he was "dismayed" by the move, adding: "I think he is a big loss to the government but I don't share his judgement."
Business Secretary Lord Mandelson said he "regretted" Mr Purnell's decision to quit but said he had done so because "he did not like the face of the man at the top" rather than through any policy differences.
"He has made an electoral calculation and I think he has got it wrong. The rest of the cabinet is behind the prime minister," said Lord Mandelson, who added that Mr Brown was the "biggest figure in British politics to lead the country in the face of very difficult times".
Harriet Harman also joined in the criticism of Mr Purnell: "If James Purnell wants to make his decision to leave the government, then that's a matter for him, but he's not entitled to say that the prime minister has to go too, and he's not going to."
Business tycoon Sir Alan Sugar, who has been appointed an "enterprise tsar" in the reshuffle, also backed Mr Brown saying: "We are in an emergency situation as far as the economic conditions go... I can not think of a better person to be in place."
Cameron: Time for general election
One group of Labour MPs have told the BBC they may delay their plans to circulate an e-mail gathering support for Mr Brown to quit.
But some Labour backbenchers and senior figures in the party, including former chairman of the Parliamentary Party Lord Soley, have said there has to be a change of leadership.
Senior Labour MP Barry Sheerman told BBC News he wanted Mr Brown to stand down and predicted many of his backbench colleagues would vote that way if they were "liberated by a secret ballot".
Conservative leader David Cameron and Lib Dem leader Nick Clegg repeated their calls for a general election.
Mr Cameron told BBC News the government had "lost the right to govern," adding: "We have a government in complete chaos. We really do deserve better than this."
Mr Clegg said Mr Brown's future as PM was "irrelevant" because the Labour government was "finished" and had "run out of road".
"The Labour Party has no right, at a time when people are crying out for help, to hold the country to ransom with its own splits and infighting," he added.
Mr Purnell's resignation came as the polls closed on Thursday for the European and English local elections.
In a letter published in several newspapers, the work and pensions secretary said he was not seeking the leadership but told Mr Brown: "I now believe your continued leadership makes a Conservative victory more, not less likely. That would be disastrous for our country."
It comes after the resignation of Home Secretary Jacqui Smith, Communities Secretary Hazel Blears and two junior ministers.
Recriminations have already begun over the elections, with John Prescott blaming Harriet Harman, his successor as Labour's deputy leader, for running a "non-campaign" and accusing her and other cabinet ministers of being "resigned to defeat".
Labour is finished, claims Clegg
In a strongly-worded broadside on his Labour Home blog, Mr Prescott also singled out elections co-ordinator Douglas Alexander, Europe minister Caroline Flint and former Local Government Secretary Hazel Blears for heavy criticism.
John Prescott warned that the Labour Party should not blame the results solely on the expenses scandal but also on senior ministers' "dereliction of duty".
And he attacked Mr Purnell over his decision to quit the cabinet, saying he was "not so much a Blairite as a careerite".
The results of the European election, which was also held on Thursday, will start to be published from 2100 BST on Sunday.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.