Culture secretary Tessa Jowell has called for Ken Livingstone to apologise over a row in which he made a Nazi jibe to a Jewish journalist.
The mayor claims he was being harassed by the reporter
At a party last week the Mayor of London likened the behaviour of an Evening Standard reporter to a "concentration camp guard".
On Sunday Ms Jowell told GMTV he should apologise for losing his temper.
A spokesman for the mayor has accused the newspaper of harassment - a charge the Standard denies.
Ms Jowell told the programme: "I don't believe he has a racist or an anti-Semitic bone in his body - look at his political record.
"Sometimes he has a filthy temper and I think he lost his temper because he was door-stepped in an aggressive way.
"I think it is always a good idea that if you lose your temper it is good to apologise for losing your temper."
Ms Jowell said Mr Livingstone was not anti-Semitic
Mr Livingstone faces censure from the London Assembly over his outburst on Monday.
Brian Coleman, the assembly's Tory chair, has tabled an emergency motion calling on the Mayor to apologise and withdraw his comments, which will be discussed at GLA budget meeting.
"All reasonable people think the Mayor has gone too far," Mr Coleman said.
"His original remarks were deeply offensive. He then compounded the situation by refusing to apologise.
"This entire episode has been a failure of civic leadership in his behalf."
'German war criminal'
The exchange with reporter Oliver Finegold took place as Mr Livingstone left a party marking the 20-year anniversary of former Culture Secretary Chris Smith coming out as the first gay MP.
On tape Mr Livingstone is heard asking Mr Finegold if he was a "German war criminal".
Mr Finegold replies: "No, I'm Jewish, I wasn't a German war criminal. I'm quite offended by that."
The mayor then says: "Ah right, well you might be, but actually you are just like a concentration camp guard, you are just doing it because you are paid to, aren't you?"
The mayor's office has alleged Mr Finegold swore at Mr Livingstone at the end of the conversation.
They also described his actions of questioning Mr Livingstone after the party as "door-stepping" and said the conversation began as "relatively light-hearted".
The Board of Deputies of British Jews has described Mr Livingstone's comment as "insensitive".