James DeGale won gold for Great Britain at the Beijing Olympics
Wembley could host the 2012 London Olympic boxing tournament, provided the sport's authorities approve a move.
Plans to axe a proposed temporary venue would see several sports relocated.
The International Olympic Committee has backed taking boxing from East London to Wembley but there are fears it is too far from the athletes' village.
"We want the best conditions for boxers and to ensure boxing has a legacy," said Richard Caborn, President of the Amateur Boxing Association of England.
The proposal is to save £20m by not building a planned temporary venue called North Greenwich Arena 2, which would see badminton, rhythmic gymnastics and sitting volleyball relocated to the ExCel Arena.
But badminton can only be fitted in there if boxing moves to Wembley.
"They are trying not to spend money when it's not necessary," said IOC director of sport Gilbert Felli.