Sir Thomas angered MPs by imposing retrospective limits on some claims
MPs should repay £1.12m of their second home expenses, an audit of claims dating back to 2004 has said.
Sir Thomas Legg recommended that 389 MPs, more than half the current and past MPs reviewed, should repay £1.3m.
Some complained about the way he carried out the audit and £180,000 was cut off the total after appeals.
Sir Thomas said the expenses system was "deeply flawed", the rules "vague" and it had been up to MPs to "self certify" the propriety of their claims.
MPs had to sign a declaration with each claim saying "that I incurred these costs wholly, exclusively and necessarily to enable me to stay overnight away from my only or main home for the purpose of performing my duties as a Member of Parliament".
752 MPs and ex-MPs investigated
Legg asked 389 to repay money - reduced to 372 after appeals
363 not required to repay money
75 appealed against rulings
31 appeals dismissed
17 repayments reduced to zero
42 appeals wholly or partly successful
In his report Sir Thomas pointed out there had been a "culture of deference" to MPs by expenses officials and "no audit of any kind" of second homes expenses during the period he covered.
"Neither internal nor external auditors could 'go behind the member's signature'," he said.
The report said £800,000 had been repaid already - some unconnected to Sir Thomas's demands - since April 2009, the month before the scandal broke.
Sir Thomas said 389 people should repay money but this was reduced to 372 after an appeals process.
The highest amount recommended for repayment, following the appeals process, is £42,458 for Labour junior minister Barbara Follett.
It relates to claims for mobile security patrols at her second home - which Sir Thomas said went beyond what was allowed under the rules, claims for six telephone lines which he ruled was "excessive" and an insurance premium for fine art.
Ms Follett has already repaid £32,976 and told the BBC: "This has been a very sad affair, I'm very sorry about it, I did try to act as honestly as possible but where I failed, I am sorry."
Other large repayment requests were about £60,000 in total from husband and wife Conservative MPs Andrew MacKay and Julie Kirkbride and £24,878 from shadow defence secretary Liam Fox - all of whom have repaid the money although Liam Fox says his appeal is still pending.
Some MPs have criticised Sir Thomas's audit - which itself cost £1.16m - saying mistakes were made and some said their reputations were unjustly damaged.
The audit covered all MPs apart from inner London ones who were not eligible to claim the second homes allowance.
WHAT MPs MUST REPAY
£4,000 for hotel stays
£711,000 for mortgage/rent
£12,000 for food
30 £10,000 for utilities
£35,000 for 59 Council Tax/Rates
£23,000 for phone & telecoms
£105,000 for cleaning
£81,000 on service/maintenance
£252,000 - 182 other payments:
Seventy five MPs and former MPs appealed against Sir Thomas's recommendations - 31 were dismissed, 27 had the repayment reduced and 17 had the demands overturned entirely.
Conservative MP Bernard Jenkin had a request for £63,250 reduced by £27,000 while the ex-Conservative minister Peter Lilley had his demand for £41,057 overturned entirely.
The judge who ruled on appeals, Sir Paul Kennedy, said each case had to be looked at "on its own merits".
He also said he was "particularly troubled" that MPs who had not broken any rules at the time had been accused of making "tainted" claims or having "breached the requirement of propriety".
Many MPs complained that Sir Thomas had retrospectively applied limits to claims for gardening and cleaning that were not in place at the time.
Tory MP Ann Widdecombe said there had been "no faith" in the report and told the BBC: "It was illogical because Legg went back and said well we're going to have retrospective limits for gardening and cleaning but not for food and mortgages."
Deadline to repay
But Sir Thomas said the rules stated that MPs should only be reimbursed "for specific and proportionate expenditure... needed for the performance of Parliamentary duties".
Just because MPs and officials "acted in apparent ignorance" of those standards - it did not mean the payments had been valid.
He also criticised a "widespread lack of proper evidence on the record from MPs to support substantial payments" and said the second homes expenses were "deeply flawed".
"In particular, the rules were vague, and MPs were themselves self-certifying as to the propriety of their use of the allowance," he said.
Ben Wright, BBC political correspondent
Throughout the whole expenses saga there have been plenty of MPs who have felt aggrieved.
They say there was a system in place which they were encouraged to use, yet they are now being punished for doing so.
That feeling of injustice still hasn't gone away - in fact for those who have lost appeals it may be greater now than ever.
But despite this I think we're unlikely to see many - or indeed, any - MPs complaining publicly today.
Sir Thomas is clearly incredibly unimpressed with them, to put it mildly, and I think most will now decide it best to swallow their objections and try to draw a line under the entire sorry mess.
"Taken with the prevailing lack of transparency and the 'culture of deference', this meant that the [Commons] fees office's decisions lacked legitimacy; and many of them were in fact mistaken."
Sitting MPs have until 22 February to repay the amounts requested or make "firm arrangements" to do so, otherwise Commons leader Harriet Harman told MPs a "recovery process from pay or allowances" would begin.
It is thought that 76 people have not yet repaid the money - 60 of whom are current MPs.
Some MPs have gone further than Sir Thomas required and repaid sums voluntarily when details of claims broke last year.
The Labour MP Phil Hope repaid more than £42,000 on his own initiative last May - because he said his reputation with his constituents had been dealt a "massive blow" - Sir Thomas only recommended that he repay £4,365.
'Failed to clean up system'
Hazel Blears - the former cabinet minister who famously went on TV with a cheque for £13,332 after being criticised for not paying capital gains tax on the sale of two flats, which was not against the rules, was only told to repay £225 for a shelving unit.
The prime minister's spokesman said Gordon Brown was fully supportive of Sir Thomas Legg's review and that he encouraged MPs to pay back money owed as quickly as possible and it was an important step in overhauling a discredited system.
HAVE YOUR SAY
Is the row over MPs' expenses now over? Not by a long chalk
Richard Hill, Birmingham
Conservative leader David Cameron told the BBC it was "absolutely essential" that MPs pay back the money.
"Those MPs who refuse to pay it back, they should have it taken off their salaries or their redundancy payments - that's got to happen."
Lib Dem leader Nick Clegg told the BBC: "I hope it'll be the final chapter in this rotten Parliament so we can look forward to a new Parliament with new rules."
Meanwhile the Crown Prosecution Service says it will announce on Friday whether it will bring charges against MPs and peers over expenses.