Massow criticised the art of Tracey Emin
Mr Massow caused a storm in the art world last week after writing much modern art was "pretentious, self-indulgent, craftless tat".
The businessman said he was stepping down after losing the support of the board.
In an article for last week's New Statesman, Mr Massow also criticised what he called the "cultural tsars", led by Tate museum director Nicholas Serota, who, he said, "dominate the scene from their crystal Kremlins".
Mr Massow became chairman of the institute in 1999 with a view to helping the body's fundraising and strategy.
A statement from the ICA read: "Ivan Massow, chairman of the ICA, has resigned from his post with immediate effect.
'Unanimous' board
"An interim chairman will be announced within the next few days and meanwhile it is very much business as usual."
Mr Massow said: "One person said she wanted me to resign, others had criticisms... they were not prepared to come off the fence."
The businessman said he had asked the board whether they were prepared to sack him.
He said: "I was not prepared to resign voluntarily unless the board unanimously wanted me to resign.
"They all went round the table... they all wanted me to resign."
He added: "The ICA found itself in a difficult and embarrassing situation because it clearly goes against the spirit of the ICA to prevent debate.
"I made sure I did not have their confidence before resigning."
Asked about the damage the article had done, Mr Massow said: "I think they felt the chairman should not have written the article.
"They felt that the statements were incompatible with the role."
But he added: "It has been a great few years. I wish the ICA all the success in the future."
Sir Nicholas refused to comment on Massow's resignation on Tuesday as he attended the launch of Tate Modern's Warhol exhibition.