British Olympic Association chairman Lord Moynihan wants tighter financial management of the London 2012 Games.
Moynihan won Olympic silver as a rowing cox at the 1980 Games
The London Olympic Board member also says there must be more transparency around the Games' £9.3bn budget.
Moynihan told the Daily Telegraph: "We don't know what is included in the budget and what isn't - it's clear that there is a lot of room for improvement.
"These are everybody's Games so it is important we have the right controls and they should be transparent."
But a spokeswoman for the Department for Culture, Media and Sport said there was a high level of "openness within the board".
She said: "It is wrong to suggest there is a lack of financial transparency - either within the Olympic board or across the whole Olympic project.
"There is a high level of openness within the board, consistent with the obligations and demands of those providing funding and commercial considerations.
"Hugely rigorous checks are in place to make sure public money is spent wisely and to achieve the maximum effect in terms of the legacy we all want from the 2012 Games.
"The National Audit Office's most recent report in July said the budget process has been thorough and assumptions made by the Government have been informed by detailed analysis and expert advice."
There is already controversy over the cost of the main Olympic stadium.
Members of the London Assembly were told two weeks ago that the cost would be £496m, 77% more that the figure stated during the bid in 2005.
The Government argues that the new figure is the cost in 2012 taking into account inflation, VAT and unforeseen extras.
Moynihan, who was the Minister for Sport in the Conservative government of Margaret Thatcher in the late 1980s, continued: "We should be aiming to be on a par with a FTSE 100 company.
"We need a clearly defined budget, a clearly defined cash flow analysis which is regularly updated and a clear focus on the contingency and how it is allocated to specific project lines in the budget.
"These things may well be going on inside Government, but they should be transparent."