Mr Hoon accepted voters were unhappy with Dr Gibson's treatment
A senior Labour figure has said it will be "difficult" for the party to win the Norwich North by-election after polls closed in the constituency.
Ex-cabinet minister Geoff Hoon told the BBC's Question Time that the contest had been a "perfect storm" for Labour.
It is the first Parliamentary contest since the MPs' expenses scandal broke and was forced by sitting Labour MP Ian Gibson's decision to step down.
The result is expected to be declared at about 1200 BST on Friday.
A total of 12 candidates are standing in the election, where Labour is defending a majority of 5,549.
Unusually for a by-election, counting will not begin until Friday with a result due around lunchtime.
Asked about Labour's prospects, Mr Hoon said the poll was taking place at a difficult time for the government in the middle of a recession and with public feelings still running high over the expenses scandal.
"This by-election has been a perfect storm for a governing party," the former defence and transport secretary said.
He acknowledged that many constituents were unhappy about the way in which Dr Gibson had been treated.
Dr Gibson was barred from standing again by Labour's National Executive Committee over his expenses claims although his local party continued to back him.
Mr Hoon said Dr Gibson had been a "very good MP" as well as a "very talented man".
"I accept that that has not gone down well in Norwich," he said of the MP's de-selection and later resignation.
"I well understand why people who worked with him, supported him and voted for him were concerned about what happened."
Commons public administration select committee chairman, Labour's Dr Tony Wright, told BBC2's Newsnight he thought Dr Gibson had been badly treated.
"There were people in the House of Commons who did far worse things than he did.
"He was a victim of a moment when all the parties, and all the party leaders, were falling over each other to show how tough they were being."
Mr Hoon accepted the situation meant it would be "pretty difficult" for Labour to win.
"For the particular circumstances I set out we were always going to have real problems in this by-election," he said.
Labour, whose candidate Chris Ostrowski was hospitalised with suspected swine flu on Tuesday, is facing a fierce challenge from the Conservatives.
Tory leader David Cameron has visited the constituency five times to back the party's candidate Chloe Smith.
Lib Dem candidate April Pond and Green Party candidate Rupert Read are also hoping for strong showings.
Glenn Tingle is standing for the UK Independence Party and Robert West for the BNP.
Among independents standing is Craig Murray, former Ambassador to Uzbekistan, on an anti-sleaze ticket.
Laud Howling is standing on behalf of The Official Monster Raving Loony Party while Thomas Burridge is representing The Libertarian Party and Anne Fryatt is representing the None of The Above Party.
Peter Baggs and Bill Holden are also standing as independents.
The full list of candidates (in alphabetical order):
Peter Baggs (Independent)
Thomas Burridge (Libertarian Party)
Anne Fryatt (None of The Above Party)
Bill Holden (Independent)
Laud Howling (The Official Monster Raving Loony Party)
Craig Murray (Put An Honest Man into Parliament)
Chris Ostrowski (Labour)
April Pond (Liberal Democrat)
Rupert Read (Green)
Chloe Smith (Conservative)
Glenn Tingle (UK Independence Party)
Robert West (British National Party)