Mr Salmond added: "The inquiry will have the fullest powers and the most searching remit. It will be charged with laying bare the outrage of why over 100,000 Scots were denied their democratic voice."
The Electoral Commission is an independent body which, although it does not run elections, reports on all major elections.
Separately, the Scotland Office said the failures must be investigated by DRS, the company which operates the electronic counting system.
It said that the Scotland Office shared the public's concern about the high number of rejected ballot papers.
A statement said: "The independent Electoral Commission will undertake a statutory review into the conduct of this election.
"It is important that they look as a matter of urgency into delays in postal ballots, the high number of spoiled ballot papers, and the performance of the electronic counting machines."
A commission spokeswoman said she could not comment on whether the election would have to be re-run.
She added that it was not up to the organisation to re-play elections, but anyone, including members of the public and candidates, can petition the Royal Courts of Justice to dispute a result.
The prime minister's official spokesman described the matter as serious and said he shared the public's concern.
Concerns have been raised about the decision to stage the Scottish Parliament and the local authority elections on the same day.
Voters were presented with two ballot papers and different voting systems.
The Electoral Commission has launched an investigation
The local authority ballot used the new Single Transferable Vote system.
The returning officer at the Glasgow Shettleston count said there had been 2,035 spoilt papers, while in Airdrie and Shotts, Labour's majority of 1,446 was less than the 1,536 rejected ballots.
There were 1,850 spoilt papers in Glasgow Baillieston and 1,736 in the Anniesland constituency.
The counts in Aberdeen, Argyll and Bute, Edinburgh, Eastwood, Perth and Tayside North and Strathkelvin and Bearsden were suspended until later on Friday due to technical problems.
The problem at the Strathkelvin and Bearsden count occurred when the computer system could not validate the votes that had been counted so far.
BBC Scotland political editor Brian Taylor described the situation as a disgrace.
Sonya Anderson, head of elections for DRS which implemented the automated counting system, said there was a problem with the "consolidation" of the votes.
She said: "As we scan the ballot papers we are capturing the votes and recording the fact we are capturing those votes.
"Once we have recorded all those votes we need to use a calculation to pull those results together and produce the results the returning officer is going to declare from.
'Totally inadequate'
"That process is taking longer than anticipated in some of the centres and some returning officers have decided because of the long wait that the staff and the candidates are experiencing that they are going to ask staff and candidates to go home while we resolve the issue."
During his acceptance speech as the new MSP for the Gordon constituency, SNP leader Alex Salmond criticised the voting arrangements and also earlier problems with postal voting.
He said: "The postal voting arrangements for this election across Scotland were totally inadequate.
"It is also the case that the decision to conduct an STV election at the same time as a first-past-the-post ballot for the Scottish Parliament was deeply mistaken.
"As a direct result, tens of thousands of votes across Scotland have been discounted. That is totally unacceptable in a democratic society."
David Henderson, who worked as an information officer at Thornwood Primary School in the Glasgow Kelvin constituency, said he had "no doubt" that the decision to hold two elections, under different voting systems, on the same day, had led to the high number of spoilt votes.
In a response to Brian Taylor's blog, he wrote: "I would say 60% of people coming into our polling station were unsure of how to vote.
"I'm sure the electoral commission will find that spoilt ballots were due to either to 'x's' or '1's' and '2's' on the wrong ballot paper."
Alex Donnelly, a voter in the Airdrie and Shotts constituency, said many voters had been told to fold their ballot papers despite national advice to the contrary.