"If I didn't think I was the right person, leading the right team, I would not be standing here."
Gordon Brown has unveiled a reshuffled cabinet and vowed to "fight on" with his "resilient" team to rescue the economy and clean up politics.
He admitted Labour had suffered "a painful defeat" in Thursday's polls but added: "I will not waver. I will not walk away. I will get on with the job."
And he unveiled Glenys Kinnock as Europe minister in a surprise move.
Two more cabinet ministers - Geoff Hoon and John Hutton - have stepped down but neither backed a challenge to the PM.
NEW JOBS: Alan Johnson - Home secretary Andy Burnham - Health Yvette Cooper - Work and pensions Bob Ainsworth - Defence John Denham - Communities Liam Byrne - Chief Secretary to the Treasury Ben Bradshaw - Culture Lord Adonis - Transport Peter Hain - Welsh Office Glenys Kinnock - Europe (non-Cabinet post) Sir Alan Sugar - Enterprise tsar (non-Cabinet post)
QUITTING: John Hutton James Purnell Jacqui Smith Hazel Blears Geoff Hoon Paul Murphy Caroline Flint(Minister of State) Tony McNulty(Minister of State) Margaret Beckett(Minister of State)
Speaking at a Downing Street media conference, Mr Brown said the current political crisis, fuelled by the Westminster expenses scandal "is a test of everyone's nerve - mine, the government's, the country's".
He added: "If I didn't think I was the right person to lead these challenges I would not be standing here.
"I have faith in doing my duty... I believe in never walking away in difficult times."
Mr Brown defended Chancellor Alistair Darling as a "very good personal friend" and said the idea that the pair were split over the economy was "ridiculous".
Three new ministerial councils - the Democratic Renewal Council, the Domestic Policy Council and an enhanced National Economic Council - would report weekly to the cabinet, said Mr Brown.
But speculation about his future continued as Labour MP Ian Gibson said he was standing down to force a by-election in Norwich North - and said he thinks Mr Brown's days "are close to being numbered".
BBC Political Editor Nick Robinson said if ministers had followed James Purnell in calling for Mr Brown to quit "we would today have had a new prime minister".
Several of the women attending Cabinet - myself included - have been treated by you as little more than female window dressing
But the PM had still been forced to carry out an "emergency reshuffle based on his own personal survival", which had left him "not that much stronger".
In other moves, Alan Johnson becomes home secretary and Andy Burnham succeeds him at health.
Caroline Flint has quit as Europe minister, with Glenys Kinnock, wife of former Labour leader Neil Kinnock, who recently stood down as an MEP as her successor - she will be appointed to the House of Lords to be able to take the job.
Ms Flint is understood to have quit after Mr Brown failed to promote her to a full cabinet job.
In her resignation letter she launches a stinging attack on Mr Brown for allegedly treating women ministers "like female window dressing" and running a "two tier cabinet".
She said Mr Brown had "strained every sinew" of her loyalty to the government.
There has been speculation all week that Ms Flint was about to quit, yet she had taken to the airwaves to defend Mr Brown and declare her loyalty to him.
Labour's deputy leader, and minister for women and equality, Harriet Harman: "I can understand the frustration of any woman in politics, but I don't accept that Gordon doesn't take women in politics seriously - not at all."
Alan Johnson: "To be in this position is a privilege"
Lord Mandelson's role has been expanded, giving him responsibility for higher education and training.
In what Nick Robinson said amounted to a deputy prime minister's role, he has also been given the titles of First Secretary of State and Lord President of the Council.
Mr Hoon has agreed to be the prime minister's European policy adviser ahead of the climate change talks in Copenhagen at the end of the year.
Bob Ainsworth becomes defence secretary and Peter Hain returns to the cabinet in his old job of Welsh Secretary. Ben Bradshaw joins the cabinet for the first time as culture secretary and Lord Adonis takes over at transport.
Universities Secretary John Denham succeeds Hazel Blears as communities secretary and Yvette Cooper replaces Mr Purnell as work and pensions secretary, with Liam Byrne replacing her as Chief Secretary to the Treasury.
Margaret Beckett and Tony McNulty are amongst those leaving the cabinet, the latter quitting his job as employment minister in an effort to clear his name over allegations about his expenses claims which are being investigated by the standards watchdog.
John Hutton earlier quit as defence secretary and James Purnell quit on Thursday as work and pensions secretary - but no ministers have so far backed Mr Purnell's call for Mr Brown to "stand aside".
Many Labour backbenchers who were ready to call for a change of leader will now be asking themselves: 'If they're not willing to act to end this, why should I?'
But although no cabinet ministers have backed Mr Purnell, several backbench Labour MPs have continued to call for him to stand down.
Dr Gibson - stripped of the right to stand for Labour at the next election because of his expenses - said he would stand down now to trigger what is likely to be a potentially difficult by-election for Labour.
With the majority of results now in, the scale of Labour's defeat at the English local elections is also clear after it lost control of its four remaining English county councils.
According to the BBC's projected share of the national vote at a general election, based on the English local election results in so far, the Conservatives would poll 38%, the Lib Dems 28% and Labour would be third on 23%.
Cameron hails Tory election gains
Conservative leader David Cameron said it showed his party was on course to win the next general election, adding that Labour had "lost the right to govern".
"We have clearly won this election and turned in some remarkably good results," he told BBC News.
Mr Clegg said Mr Brown's future as PM was "irrelevant" because the Labour government was "finished".
Labour is braced for another poor performance when the results of European elections are announced on Sunday.
And, in sign of the continuing febrile atmosphere, Nick Robinson said Gordon Brown had been forced to defend his expenses again over suggestions in the Daily Telegraph he claimed for electricity bills and service charges on two properties between 2005 and 2007.
A 10 spokesman insisted Mr Brown had complied with the rules at all times, a fact backed up by the Commons authorities, but he had agreed to repay about £180 "for the avoidance of doubt".
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.