Conservative Party leader David Cameron thanked officials in his Witney constituency for a well-run election day, but said that had not been the case in some parts of the country.
"An early task for a new government is to get to the bottom of what has happened and make sure that it never happens again," he said.
Liberal Democrats leader Nick Clegg, who went to speak to voters who had been waiting in long queues at a polling station in his Sheffield constituency, said he shared the "bitter dismay" of those who were unable to vote.
"That should never, ever happen again in our democracy," he said.
BBC political editor Nick Robinson said: "What a tragedy that, after a campaign which engaged and energised many who were previously cynical about politics, tonight's story may be being over-shadowed by the extraordinary revelation that Britain cannot competently run the most basic part of the democratic process."
Many voters contacted the BBC to say they had been denied the chance to vote because the polls closed while they were still queuing.
Jenny Watson, Electoral Commission: "The law on voting is extremely clear"
Jo-Ann Stranger, who failed to vote in Hackney South and Shoreditch, said it was an "outrage".
"I, along with about 100 other people, were turned away from our local polling station, having waited 45 minutes to vote.
"The officers closed the door on the queue at 9.50pm, refusing entry to anyone standing outside," she said.
"It is an outrage in a civilised country like ours that this can happen, especially to people who are trying to vote after doing a day's work.
"Whether it will affect the vote, I don't know, but the election cannot stand, given the number of voters potentially unable to vote this evening."
Kathy Murray, from Withington, Manchester, said: "I'm fuming. I queued for over an hour and had the doors shut on me, along with about 250 others, at 10pm.
"We initially went at 6pm, but it was too busy so we went back an hour later, but there were still big queues.
"We went back at 9pm but after waiting for an hour, we still couldn't vote."
In Sheffield, returning officer John Mothersole apologised to voters who were turned away, saying: "We got this wrong."
He said the turnout had been "phenomenal... probably the highest turnout in 30 years".
Deputy returning officer Lee Adams said about 200 people were turned away at Ranmoor and police dealt with a crowd of approximately 100 angry would-be voters.
She apologised and admitted staff struggled to cope, adding: "We increased the staffing at about 4pm when the staff told us they needed help.
These people have a right to sue - they will get at least £750 in my view
Human rights lawyer Geoffrey Robertson QC
"Unfortunately, the combination of the numbers turning up and the fact that many were students and didn't have their polling cards just meant it took longer to process."
But one presiding officer, Nick Baldev, laid some of the blame on voters, telling the BBC: "Many did not have their polling cards, which significantly adds time. Some people went to the wrong polling station.
"And... the absolute laziness from the elector by not joining queues when they arrive, returning at a later time only to find a longer line and re-returning at 2200 BST, which, as it clearly states, is closing time."
However, there was good news for some voters stuck in queues as the deadline approached.
At one polling station in Lewisham, south-east London, polling cards were handed out to people queuing, while at two stations in Newcastle those in the queue were ushered into the building before the doors were shut at 2200.
The Electoral Commission said each returning officer decided the number of polling stations in their constituency and the number of electors allocated to each station.
"There should have been sufficient resources allocated to ensure that everyone who wished to vote was able to do so," a statement said.
It added that anyone affected should e-mail them at email@example.com as soon as possible, and that it aimed to publish an interim report by the end of next week. Its full report is due to be published in July.
At the Ranmoor polling station in Mr Clegg's Sheffield Hallam constituency, many students were unable to vote because they were put in a separate, slower queue to other members of the public, the National Union of Students (NUS) has claimed.
The NUS said the queue containing non-students was "fast-tracked".
"What message does this send to first time voters whose votes will not be counted?" said NUS national president Wes Streeting.
Anyone wishing to contest the result in their constituency because of people not being able to vote needs to apply to the Election Petitions Office.
John Mothersole said the council was "caught out" by high turnouts
The legal application costs £5,000 and the case would ultimately be heard by two High Court judges, who could rule that another election must take place.
If they rule in your favour, you would get the £5,000 back, if not, then you would lose the money.
Human rights lawyer Geoffrey Robertson QC has also said that people denied the right to vote could sue for compensation.
"These people have a right to sue," he said.
"They will get at least £750 in my view. Under the European Convention you have a right to vote.
"They were terribly disappointed, they should all sue and get money from the election commission, which seems to have incompetently overseen it."
Civil liberties group Liberty is asking anyone who was not able to cast their vote to contact its organisation.
"Liberty will use all legal and campaigning means to ensure that this disgrace is never repeated," said director Shami Chakrabarti.
"This is particularly pressing given the possibility of further elections in the medium term."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.