Barnevik engineered the creation of ABB
His departure is shrouded in criticism over his pension benefits from the Swiss-Swedish industrial group ABB where he was chairman until November last year.
Mr Barnevik said he would step down immediately "in view of the current situation".
ABB is widely seen as the flagship of the Investor network of Swedish multinational companies which it controls through minority stakes.
Jacob Wallenberg, a senior figure in the "Royal Family of Swedish Business", said Investor had "reached an agreement today with Percy Barnevik that he resigns".
Investor's executive vice chairman, Claes Dahlbaeck, has taken over as Investor's new chairman.
ABB's problems
Earlier this week, ABB asked both Mr Barnevik and former chief executive Goeran Lindahl to return some of their retirement pay, describing them as excessive in the light of the company's recent problems.
On Wednesday, ABB announced a net loss of $691m (£482m) for 2001.
The company's bottom line has been hit hard by ballooning provisions for asbestos liabilities, relating to a US subsidiary.
Mr Barnevik oversaw merger between the Swedish Asea and the Swiss Brown Boveri in the late 1980s.
The successful completion of that merger, one of the largest ever between companies based in different countries, sealed Mr Barnevik's reputation as a legend among corporate executives on a par with General Electric's Jack Welch.
Barnevik will also soon depart as chairman of Swedish tool maker Sandvik.
He is also the chairman of Anglo-Swedish pharmaceutical company AstraZeneca and he is a member the board at General Motors.