Werner Seifert, the chief executive of Deutsche Boerse, is to leave the company with immediate effect.
Mr Seifert's attempt to buy the LSE had been unpopular
The group also said that Rolf Breuer, the chairman of its supervisory board, would depart at the end of this year.
Deutsche Boerse made the resignation announcements following a meeting of the supervisory board on Monday.
Mr Seifert's position had weakened after his failed attempt to buy the London Stock Exchange, a move strongly opposed by some shareholder groups.
Shares in the Deutsche Boerse were temporarily suspended following the announcement of Mr Seifert's departure.
When trading resumed, they closed up 2.5% at 61.97 euros.
Mr Breuer will lead the search for a new chief executive, while chief financial officer Mathias Hlubek will fill the position on an interim basis.
Deutsche Boerse added that three further members of its supervisory board are to leave, but did not give their names.
Its planned £1.3bn takeover of the London Stock Exchange (LSE) fell through on 7 March.
The German company said at the time that it had failed to convince the LSE to recommend the offer, and also acknowledged widespread opposition among its own shareholders.
"The decisions to change the supervisory and executive boards...were taken after extensive consultation with shareholders of the company," Mr Breuer said.
"In the interest of the company we want to put an end to the recent debate with some of our shareholders."
Shareholder opposition to the LSE bid had been led by UK hedge fund TCI.
It had put forward a motion to Deutsche Boerse's annual general meeting on 25 May to remove Mr Breuer.
'Responding to concerns'
A number of shareholders welcomed Mr Seifert's decision to resign.
"If you look at the reaction of the stock, Mr Seifert has created more value with the announcement of his resignation than he did with any other corporate action," Daniel Loeb, an analyst with Third Point - a shareholder in Deutsche Boerse - told Reuters.
US investment firm Atticus, which owns about 5% of Deutsche Boerse shares, said the executive departures "reflected well" on German corporate governance.
"It shows responsiveness to shareholder issues," Rob Coburn, its director of marketing and communications, told Reuters.