A Jewish leader has demanded Ken Livingstone be brought to account after accusing a reporter of behaving like a "concentration camp guard".
The mayor claims he was being harassed by the reporter
The exchange with an Evening Standard journalist happened as the mayor left an event marking 20 years since Chris Smith came out as the first gay MP.
The Mayor of London's office defended him saying he had been harassed at a "predominantly gay event".
The Board of Deputies of British Jews said the comment was "insensitive".
On tape the mayor is heard asking Oliver Finegold if he is a "German war criminal".
Mr Finegold replies: "No, I'm Jewish, I wasn't a German war criminal. I'm quite offended by that."
The mayor then says: "Ah right, well you might be, but actually you are just like a concentration camp guard, you are just doing it because you are paid to, aren't you?"
The mayor's office alleged Mr Finegold swore at Mr Livingstone at the end of the conversation.
They also described his actions of questioning Mr Livingstone after the party as "door-stepping" and said the conversation began as "relatively light-hearted".
Henry Grunwald, president of the Board of Deputies of British Jews told BBC News: "I find his language appalling and this man's insensitivity appears to know no bounds at all.
"I couldn't begin to fathom what drives this man in the language that he uses but it really is about time that he was brought to account for that language and I would ask that the board for standards in local government considers this latest outburst.
"Over the last two weeks the nation has recognised what happened in Auschwitz 60 years ago.
"Two weeks ago we marked the liberation of that camp by the Allies and British survivors of the Holocaust and members of the British armed forces who liberated camps like Belsen were at the centre of Holocaust memorial day.
"And against that backdrop, for a man like Mr Livingstone to use language that he did and the way that he did shows remarkable insensitivity and unthinking."
The gay rights group Outrage has said the mayor should not "embroil lesbians and gay men in his dispute with the Evening Standard" and that it would not fear a journalist reporting on Tuesday's party.
The chair of London Assembly Brian Coleman is calling for the mayor to apologise for his comments.