Tory leader David Cameron has called London Mayor Ken Livingstone an "ageing far left politician" hanging onto outdated views about ethnic minorities.
Mr Cameron said the Tories had to change
London's mayor saw them as "potential agents of revolutionary change", not equals wanting a better life, he said.
It follows the mayor's attacks on calls for multiculturalism to be ditched.
The mayor said Mr Cameron's view was just a "softer version of the Tebbit test" whereby people must lose their cultural identity to be truly British.
Earlier this week the mayor of London boycotted an event organised by Mr Phillips, head of the Commission for Racial Equality, as part of an ongoing row.
He recently suggested Mr Phillips, who said that the term multiculturalism implied "separateness" and should be dropped, had "gone so far over to the other side that I expect soon he'll be joining the BNP".
On Wednesday, in a speech to the Ethnic Media conference, Mr Cameron said "any serious conversation" about racism must move beyond "old Marxist clichés".
Mr Cameron said: "Insulting Trevor by saying he should join the BNP isn't a serious contribution to debate.
"It's a discreditable attempt by an ageing far left politician to hang on to a narrative about race that sees people from ethnic minorities as potential agents of revolutionary change."
He added that they should be seen as "full and equal citizens who would rather build a better life for themselves and their families than man the barricades at the behest of middle class white fantasists".
Mr Cameron went on to say that he wanted to "broaden the base" of the Conservative Party by encouraging more black and Asian people to get involved.
He said he would introduce the monitoring of their progress within the Conservative party - from local council and parliamentary candidates, to people employed by the party.
"I know that monitoring makes some uneasy, but if we fail to find how well we as a body are doing, we have no way of remedying the situation," he said.
The mayor has clashed with Mr Phillips several times
There would also be a road show in British cities in partnership with Operation Black Vote to encourage more political participation among ethnic minority communities.
And the party would offer internships for 20 young black and Asian people a year to work at Conservative headquarters, or in Parliament.
Mr Cameron said that, in the past, the Tories had thought it did enough to provide equal opportunities - but that was not the case.
'Attack on multiculturalism'
"The fact is that it's not enough to open the door to ethnic minorities. If people look in and a see an all-white room they are less likely to hang around.
"An unlocked door is not the same as a genuine invitation to come in."
Later Mr Livingstone said Mr Cameron's comments showed his opposition to multiculturalism - which he said was one of the keys to London's success as "the greatest city in the world".
"Every member of an ethnic minority, of any income, class or creed, now knows that they are expected by him and his colleagues to abandon their cultural identity to be really considered British citizens. It is simply a softer version of the 'Tebbit test'," he said.
"London's internationalisation and multiculturalism are the keys to its success as the greatest city in the world. These have allowed it to overtake even New York as the most successful international city.
'Economically Mr Cameron's attack on multiculturalism would threaten the international character of London that gives it an advantage over its competitors - thereby undermining the prosperity of the city."