The prince is said to have used the nickname for many years
An Asian polo club member referred to by the Prince of Wales as "Sooty" has said Charles has "zero prejudice".
Kolin Dhillon said the nickname used by his friend was a "term of affection with no offence meant or felt".
He said it was also used by other members at the Cirencester Park Polo club in Gloucestershire.
The story emerged days after Prince Harry apologised for using the word "Paki" to describe a member of his army platoon in 2006.
Clarence House refused to comment on the polo club story but said suggesting Charles was racist was "completely ridiculous".
A spokesman said they were not going to comment on the use of an alleged nickname at a private club.
He added: "Look at the work he's done here and abroad. Consistently over 30 years he's called for greater tolerance and understanding."
Mr Dhillon, a former chairman of the Schools and Universities Polo Association originally from the Punjab, emigrated to Britain in 1955.
In a statement issued from his Cheltenham office, the property developer said: "I have to say that you know you have arrived when you acquire a nickname.
"I enjoy being called Sooty by my friends who I am sure universally use the name as a term of affection with no offence meant or felt.
"The Prince of Wales is a man of zero prejudice and both his sons have always been most respectful."
Mr Dhillon's full name is Kuldip Singh Dhillon but he is said to be known under the Anglicised name Kolin.
A member of Cirencester Park Polo Club spoken to by the BBC said the nickname used for Mr Dhillon had not caused any offence.
He said: "I know the Prince calls him Sooty. It's not a problem on either side."
Cirencester Park Polo Club declined to comment on the reports.
But leading British-Asian businessman, Sir Gulam Noon, who has worked with Charles on many occasions, said there was no suggestion the prince had done anything wrong.
"Political correctness has gone a little bit too far," he told the BBC.
He added: "I have been working with Prince Charles for more than 20 years now and he has always been absolutely courteous and respectful to every Asian community member he has ever met."
But the Labour MP Keith Vaz, chairman of the Home Affairs Select Committee, told the BBC: "I do worry about the choice of these nicknames which they regard as terms of endearment and affection but which members of the public will regard as being offensive and distasteful."
BBC royal correspondent Nicholas Witchell said the nickname clearly did not offend Mr Dhillon.
However, our correspondent said people may consider its use harks back to a time when racial attitudes were offensive and out of tune with today's Britain.
On Sunday, Prince Harry issued an apology after the News of the World published a video diary in which he calls one of his then Sandhurst colleagues a "Paki".
St James's Palace said he had used the term about a friend and without malice.