Progress in tackling racism on the football pitch and the stands is not being mirrored at the top of the game, according to a new report.
Despite progress there is no room for complacency, says the report
Boardrooms at every club level remain almost wholly white and male, the Independent Football Commission found.
English football authorities need to be "radically restructured" to better reflect society, the IFC said.
But achievements such as integrating black players are praised.
Skilled people from ethnic minorities should be co-opted onto the sport's governing bodies, the IFC believes.
The slow pace of change is damaging perceptions of the game, it found.
Asian communities are not adequately represented in football, with people from ethnic minorities lacking the opportunity to reach coaching, scouting and management levels.
The report's author Professor Derek Fraser told the BBC more needed to be done.
Places should be set aside on the FA Premier League and Football League boards this year, and further targets introduced for the next three years, said the commission.
'Could do better'
It also calls on the FA Premier League and Football League to publish strategies for better racial integration in the sport.
"We think some direct action such as co-option would be giving a signal to the ethnic minority communities who feel there are still barriers
against wider participation in the game," said Prof Fraser.
But he added: "I think we're ahead of the game."
"We're doing well but we could be doing even
The report concludes: "Too little is being delivered and too slowly."
Despite praising initiatives by clubs such as Leicester, Leeds and Southend, the commission called for "positive moves to attract more non-white supporters to grounds".
Garth Crooks has led the IFC's examination of racism issues
"If only an Asian player could be seen lifting one of the major club trophies; if only more black and Asian referees stayed the course and didn't find racial comments just one set of abuse too much, and if only there was a black or Asian chairman at a Premier League club," added the IFC.
One of its member and former Premiership footballer Garth Crooks said: "For change to be meaningful it has to be radical.
"This report is not about a token black face here and there."
FA Premier League chief executive Richard Scudamore told BBC Radio 5 Live: "I can see that maybe we are not as diverse as we might be - and that is what we need to address."
FA Chief Executive Mark Palios said he was "greatly encouraged" the IFC had acknowledged their achievements but he believed there was still more to do.
The FA board decided last year to introduce co-options to its committees to bring in new talent, he said.
Football League chairman Sir Brian Mawhinney said it would consult with clubs on the issues raised.