Mr Brown accepted that anger among Labour supporters affected the result
Senior Labour MPs have accused Gordon Brown of being directly responsible for the party's crushing defeat in the Norwich North by-election.
Tory Chloe Smith reversed a comfortable Labour majority to win by 7,348 votes.
Ex-home secretary Charles Clarke blamed the result on Mr Brown's "incompetent" treatment of outgoing MP Dr Ian Gibson.
Senior backbencher Barry Sheerman said the result was a "self-inflicted wound" and warned Mr Brown had until the end of the summer to reconnect with voters.
Left-winger John McDonnell said the prime minister had made a "terrible miscalculation" in his handling of the episode while fellow Labour backbencher, Kate Hoey, told the BBC that Mr Brown needed to re-examine his leadership techniques.
She said:"By-elections are always unique but there is no doubt about it that this is a bad result."
"The prime minister I hope will be looking at how he's looking to lead the party and to talk to the party, and a lot of party members feel that they are not listened to."
But Tony Lloyd, chairman of the Parliamentary Labour Party, insisted there was "no leadership crisis taking place".
Mr Brown has acknowledged the by-election result was affected by the anger of many local Labour supporters at the party executive's treatment of their former MP.
Dr Ian Gibson had resigned after being told he could not stand for election again by the "Star Chamber", which ruled on MPs' positions in light of their expenses claims.
Sheerman urges Brown to lift morale
"The voters were clearly torn between their anger and dismay at what's been happening with MPs' expenses... [and] support for the former MP," Mr Brown said.
He claimed none of the main parties could take "any cheer" from the result, with only the fringe parties picking up more votes than previously.
However, Mr Sheerman says morale among Labour members was low and that the party was in a "desperate situation".
The MP - who had called for a secret ballot on Mr Brown's leadership after last month's dismal showing in the local and European elections, before promising to support the prime minister for the "time being" later in June - warned that Mr Brown urgently needed to demonstrate he could re-connect with voters.
"He's got this summer to show he's got the capability to do it," Mr Sheerman said.
"In any other human organisation I know, if the chief executive actually doesn't make it, he doesn't get it and he doesn't deliver, then he's got to consider his position."
Mr Clarke - the MP for neighbouring Norwich South and a long-time critic of the prime minister - said there had been no "guiding principles" to the prime minister's handling of the expenses scandal.
"What happened to Ian Gibson was not fair and many, many people felt that," Mr Clarke told the BBC.
"You need the transparency, you need a comprehensive approach, you need fairness and you need it to be done quickly and these things didn't happen."
NORWICH NORTH BY-ELECTION
Chloe Smith (Con) 13,591 (39.5%)
Chris Ostrowski (Lab) 6,243 (18.16%)
April Pond (LD) 4,803 (13.97%)
Glenn Tingle (UKIP) 4,068 (11.83%)
Rupert Read (Green) 3,350 (9.74%)
Craig Murray (Ind) 953 (2.77%)
Robert West (BNP) 941 (2.74%)
Bill Holden (Ind) 166 (0.48)
Howling Laud (Loony) 144 (0.42%)
Anne Fryatt (NOTA) 59 (0.17%)
Thomas Burridge (Libertarian) 36 (0.1%)
Peter Baggs (Ind) 23 (0.07%)
Con majority 7,348 (21.37%) 16.49% swing Lab to Con Turnout 34,377 (45.76%, down 15.33% on gen election)
Mr McDonnell, MP for Hayes and Harlington, said the leadership should apologise to Labour activists for its treatment of Dr Gibson.
He added: "Between them, Gordon Brown and Peter Mandelson decided that Ian Gibson should go.
"If they had listened to the local party members we wouldn't be in this mess."
Ms Hoey, MP for Vauxhall, said those seeking a leadership election had missed their chance.
She said there were no current serious contenders and most MPs would use their summer break to reflect.
But Mr Lloyd said the real issue for voters was the recession, adding that they would come back to Labour once government measures to boost the economy bore fruit.
"Time is beginning to tell that these policies have worked," he told the BBC's Today programme.
He added that Mr Sheerman was "one of a small group of persistent critics of Gordon" and "doesn't represent anything like even a significant view in the parliamentary party".
The by-election saw a swing from Labour to the Tories of 16.5% as they garnered more than 13,500 votes.
The Lib Dems came third with 4,803, narrowly ahead of UKIP on 4,068, who enjoyed their best ever by-election result.
Mr Cameron said the victory - only the Tories' second by-election win in a Labour seat for 27 years - showed people "want change in our country".
He accused Labour of running an "utterly despicable" campaign full of allegations about policies which were "not true".
"I say to this prime minister who talks about courage and who talks about a moral compass, where was the courage in not even coming to this by-election?
"And where is the moral compass in allowing a campaign of lies and half truths about your opponents?"
Ms Smith, who at 27 will be the youngest MP in the House of Commons, paid tribute to Dr Gibson in her victory speech and wished her Labour rival Chris Ostrowski a speedy recovery - he was hospitalised with suspected swine flu days earlier.
She will not take her seat officially until October, after the summer recess, but said she would be concentrating on her constituents.
"This isn't about me jumping off here and then going and living up on expenses for the three summer months," she said.
"This is about getting down to it, being honest, and being held to that."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.