Edgar Griffin has denied any wrongdoing
One senior supporter of Iain Duncan Smith has accused Montgomeryshire Conservative Association of failing to vet 79-year-old Mr Griffin before his name was submitted to the Tory hopeful's leadership campaign.
Dr Felix Aubel, a member of the Duncan Smith campaign in Wales, said Mr Griffin had been added to the supporters' list in good faith.
His position as vice-chairman of Montgomeryshire Conservative Association should have been scrutinised, said Dr Aubel.
"I found it absolutely incredible that the Montgomeryshire Conservative Association would select as its vice-chairman a man of such extreme, racist and neo-facist views," he said.
But the man accused of tarnishing the Duncan Smith campaign said he had "no connection whatsoever" with the British National Party.
Mr Griffins said: "However they are saying this in London, it's a nonsense.
"The Welsh are very jealous of their independence from London ... I think they will simply tell London what they can do with the rejection of Edgar Griffin.
"I have done a decent job, I have worked hard and there is no earthly reason for them to ask for my resignation and I don't think it will come to that."
But acting Tory chairman David Prior said the action had been brought about through the recommendation of the chairman of Welsh Conservatives.
Councillor David Jones, a former chairman of Montgomeryshire Conservatives, confirmed they were aware that Mr Griffin's son was the leader
of the BNP.
But until Friday, Mr Griffin had kept the BNP "at arms length", and had not expressed sympathies with them, said Coun Jones.
He acknowledged that if the Party had expelled him on Thursday, local members would have been uneasy that Mr Griffin was being punished for the actions of his family.
But Mr Jones said the remarks made by Edgar Griffin on radio and television were wholly unacceptable, and that he expected little sympathy for Mr Griffin among other members of the Conservative Association.