Mr Ross is also on the board of the organising committee for London 2012
David Ross, co-founder of Carphone Warehouse and holder of 19.4% of its shares, has resigned as a director in a scandal over the use of his shares.
Mr Ross used 136 million of his 177 million shares as security against personal loans without telling anyone in the company.
He has admitted doing the same thing with shares of Big Yellow Group and National Express.
Mr Ross is also on the board of the organising committee for London 2012.
He was London Mayor Boris Johnson's appointment to the board of London 2012, charged with making sure that the games did not go over budget.
He stressed that he had no intention of selling the shares and none of the loans was in default.
At Friday's closing price, Mr Ross's total stake in Carphone was worth just under £165m.
Mr Ross is also chairman of National Express Group, and has informed the company that he used a total of £14m worth of its shares to secure personal loans.
He did the same thing with his £24m shareholding in self-storage company Big Yellow Group, of which he is a director.
Big Yellow Group said that Mr Ross had only just informed the firm.
There was no indication that he would resign from his posts at National Express or Big Yellow Group.
Mr Ross is a member of the family that started the Grimsby-based Ross frozen fish brand, which is now owned by Young's frozen foods.
Carphone Warehouse chief executive Charles Dunstone said the resignation was a "great personal sadness". The two were at school together.
He said the incident was "probably an oversight or a misunderstanding of what needed to be done".
Stock exchange rules say that if a director uses his shares in this way, the company has to make an official announcement.
The company made such announcements in 2002 and 2003, when Mr Ross used some of his shares as security for personal loans.
Mr Ross and Mr Dunstone are by far the biggest shareholders in Carphone Warehouse, with 52% between them.
No other shareholder or fund has more than 4%.
BBC business editor Robert Peston said that the announcement would raise uncertainties about the future ownership of one-fifth of the company.
In afternoon trading, Carphone Warehouse were down 3.2%, Big Yellow Group shares were down 7.9%, while National Express shares were down 2%.