David Cameron has accused Gordon Brown of "treating the British people as fools" in not calling a snap election.
Mr Brown told the BBC he had had a "duty" to consider whether to hold an election, but decided against it so he could show his "vision" for Britain.
But the Tory leader said the PM "was not being straight... everybody knows he is not having an election because there's a danger of him losing it".
Lib Dem leader Sir Menzies Campbell accused Mr Brown of a "loss of nerve".
Mr Brown's announcement to the BBC's The Andrew Marr Show that there would not be an election came as a poll published by Sunday's News of the World suggests the Tories are ahead by 6% in marginal seats, with the party overall at 44% against Labour's 38%.
Translated into a general election, it would mean a hung Parliament with Labour holding 306 seats and the Tories 246.
A poll carried out for the Sunday Times suggested the Tories had taken a three-point lead.
During the interview Mr Brown also said it was "very unlikely" there would be an election in 2008.
He denied the opinion polls had led to the decision not to hold an election, saying: "I have a vision for change in Britain and I want to show people how in government we're implementing it."
Pressed on the decision, Mr Brown said that the series of crises since he became PM in June meant "the easiest thing I could have done is call an election. I could have called an election on competence".
He added: "We would win an election, in my view, whether we had it today, next week or weeks after."
But, he said: "I want the chance in the next phase of my premiership to develop and show people the policies that are going to make a huge difference and make a change in the whole country itself."
He accepted that the widespread speculation he was considering holding an election had been true.
"When you are prime minister and you have the power to make this decision, people come to you and say 'you must consider it' and you've got to consider it because you have to exercise power with responsibility," he said.
That responsibility also included listening to the British people - and he said they wanted to see him "get on with the job" rather than holding an election.
Mr Brown said he "relished" the chance to get stuck into the Conservative policy pledges on issues like inheritance tax.
But Mr Cameron said: "The reason the prime minister has cancelled this election is that the Conservative Party is making the arguments about the changes this country needs. People are responding very positively to our proposals."
Shadow Chancellor George Osborne said: "The Labour Party promised us at the last election that Tony Blair would serve the full term and then there would be an election.
"Well, Tony Blair's gone, the new guy's in charge, he's been there for a while now. He's deliberately stoked up speculation about an election and now he's gone and bottled it."
Lib Dem leader Sir Menzies Campbell said his party would table a bill in the House of Commons this week proposing a fixed time between elections.
He said: "It's high time, in my view, that the decision about a general election is not part of the discretion of the prime minister.
"Inevitably in those circumstances the prime minister's attitude will be conditioned by what he thinks is best for him and best for his party."
Scottish National Party leader, and Scottish First Minister, Alex Salmond told Sky News: "Authority, credibility in politics is a strange thing. You can take 10 years to build it and you can lose it in a single day."
Labour's deputy leader Harriet Harman conceded on BBC Radio 4's The World This Weekend the episode could have damaged Mr Brown.
Asked if it had damaged him she said: "I don't think it should damage him at all."
She was then asked: "So it may have done?" She replied: "Well, you know we'll see."