Byron Butler made his comment when interviewed by Ian 'H' Watkins
A magistrate has agreed not to hear cases as an inquiry gets under way into comments he made about gay men in a BBC Wales documentary.
Byron Butler is being investigated by the advisory committee on Justices of the Peace in Mid Glamorgan after being interviewed by Week In Week Out.
In the programme, the Bridgend JP said many thought gay men were paedophiles.
The programme, hosted by former Steps singer Ian "H" Watkins, looked at changing attitudes to homosexuality.
Mr Butler, 67, a magistrate for 30 years, is a former deputy lord lieutenant for Mid Glamorgan, who would have deputised for the Queen's official representative in the county.
He was one of many people interviewed in the street for his views on homosexuality by Watkins, who grew up in the Rhondda.
Mr Butler, a retired butcher, said: "We haven't got much time for them."
He continued: "Well, I think probably it's a suspicion of the mainstream that they perhaps will interfere with young people and so on and that's historically been the case. That is the danger.
"Paedophiles, solidophiles [sic], not necessarily, but they do, don't they. That's the reality."
A complaint about his comments was made to the Office of Judicial Complaints after the programme was screened.
He has voluntarily decided not to sit as a magistrate after an investigation was started into the matter.
The programme, which was broadcast last month, featured Watkins returning to his home area of Cwmparc to make the documentary, called The Only Gay in the Village.
Watkins, a former Celebrity Big Brother contestant, spoke about his life growing up gay in the Welsh valleys, which he called "a complete nightmare".
Following the remarks by Mr Butler, the singer called for him to be stripped of his JP's role.
He said: "I was dumbfounded by his comments - and I was even more shocked when I found out he is a magistrate sitting in judgement on people.
"There is no way I believe he can be fair and impartial with people when he makes comments about gay people being paedophiles. It is just so outrageous.
"There is absolutely no evidence to back up what he says and it is very offensive. The world has changed and Wales has changed so much."