Senior Conservatives have criticised comments by one of their Parliamentary candidates that Enoch Powell "was right" on immigration.
Mr Hastilow's comments were made in a local newspaper
David Davis said the comments by Nigel Hastilow in a newspaper column were "very unwise" and George Osborne said he would have to "explain himself".
Mr Hastilow has refused to comment following a meeting with party chairman Caroline Spelman on Sunday.
Labour minister Peter Hain said it showed the Tories' "racist underbelly".
Enoch Powell was sacked from the Conservative shadow cabinet after a controversial speech in 1968 against uncontrolled immigration.
Mr Hastilow, the Parliamentary candidate for Halesowen and Rowley Regis, made the comments in a column for the Express and Star newspaper in Wolverhampton - where Mr Powell had been the MP at the time of his 1968 speech.
The candidate, a former editor of the Birmingham Post, wrote: "When you ask most people in the Black Country what the single biggest problem facing the country is, most say immigration.
"Many insist: 'Enoch Powell was right'. Enoch, once MP for Wolverhampton South-West, was sacked from the Conservative front bench and marginalised politically for his 1968 'rivers of blood' speech, warning that uncontrolled immigration would change our country irrevocably.
"He was right. It has changed dramatically."
Asked about Mr Hastilow's comments earlier, shadow chancellor Mr Osborne told BBC One's Andrew Marr Show: "I don't agree with what he said."
He said Mr Hastilow would have to "explain himself" when he met the party chairman.
"It will also be explained to him that candidates of any party - Conservative, Labour, Liberal Democrat - have to exercise great caution in the language they use about immigration," he added.
He said the country needed controlled immigration, but the debate had to be conducted in "a reasonable way with moderate use of language".
And speaking to ITV1's Sunday Edition, shadow home secretary Mr Davis said the comments were "very unwise" and the constituency party should "think very hard" about how they expected their candidate to behave.
"You cannot just stumble around throwing out comments which are insensitive or inflammatory," he said, saying it would be "very sensible" if Mr Hastilow were to withdraw the remarks.
But Mary Docker, chairwoman of Mr Hastilow's local Conservative association, said she did not think he had done anything wrong.
She told BBC News 24: "He's basically just raising issues that have been raised with him when he has been canvassing the area.
"All he is doing is just relaying the views of the public, which is what a politician should do."
Mrs Docker said she did not think his comments should threaten his candidacy.
Peter Hain, Secretary of State for Work and Pensions, said the Conservatives should "get rid of this candidate" to prove they are committed to equal opportunities.
"This Conservative candidate really exposes the racist underbelly of the Tory party," he told the BBC.
He said leader David Cameron had "been very clear himself" in being against racism, but "in the undergrowth of the Tory Party...there are all these backwards reactionary sentiments".