A row has broken out over an accusation by Ceredigion Council's leader that a British National Party branch was emerging in the county.
A petition raised 8,500 votes in support of the referendum
During a debate on when to hold a referendum over a directly-elected mayor for Ceredigion, council leader Dai Lloyd Evans accused campaigners of right-wing political tendencies.
Simon Brooks of the Welsh language protest group Cymuned, one of the groups campaigning for the elected mayor, responded with counter-accusations from the public gallery in county hall in Aberaeron.
The row overshadowed the decision that the referendum will now be held on Thursday 20 May.
If people vote yes, it will be the first directly-elected mayor in Wales, with the election itself probably taking place on 21 October.
Speaking to BBC Wales News website after the debate, Mr Evans said: "All I said was that the small group of people that campaigned for the petition for a referendum had been thrown out of major political parties.
"I then said that I wondered whether a branch of the BNP was emerging in Ceredigion."
Mr Brooks said he was outraged by Mr Evans' comments.
"In effect, Dai Lloyd Evans is saying that the 8,500 people that signed the petition calling for a referendum for mayor are fascists.
"Comments like these clearly show that we have a council in Ceredigion that is openly dismissive of its own electorate.
"This is not the sort of political debate that the people of Ceredigion deserve."
Leaders of all the political parties at Tuesday's meeting said they would oppose having an elected mayor for Ceredigion.
This included the governing group, made up of the Independents and Liberal Democrats, the main opposition group Plaid Cymru and the sole Labour councillor.
The assembly was forced into action after grassroots campaigners collected 8,500 names on a petition calling for the referendum.
It followed protests in Ceredigion against plans to build 6,500 new homes in the county.
Campaigners claim the houses will destroy the local environment, damage the Welsh language. They say too many of the planned properties are executive-type developments, which local people cannot afford.