An official with one of Wales' top male choirs, Morriston Orpheus, says people "living in the past" forced its woman conductor to quit.
Sian Pearce was appointed after the former conductor stood down
But John Betts, secretary of the Swansea Valley-based choir, said Sian Pearce's gender was a "red herring".
After three years in the post Ms Pearce claimed she had been "hounded" out and some people had had a problem with a woman in charge.
She had made history as the first woman to direct the choir.
Mr Betts, giving the choir's reponse for the first time, told listeners to BBC Radio Wales: "Unfortunately some people don't want to move on, they appear to want to live in the past."
The general secretary said a small number of the choir's "disaffected members...who failed to get their way by democratic means, have made life so unpleasant for Sian, and those who actively support her, her position has been made untenable."
In her resignation letter, leaked anonymously to the South Wales Evening Post, Ms Pearce said her appointment in 2004 had been "a dream come true".
She also said: "some people have been reluctant for the choir to change, to move forward" and, in the end, she had been "hounded" out of the job she loved.
Ms Pearce's resignation letter also claimed some people did not like the fact a woman was leading the choir.
But Mr Betts said he did not believe gender had been the real issue.
"Personally. I think she has conducted the choir as well as she possibly could have done," he said.
"I just feel it's a group of people who are not prepared to move on and perhaps have used the gender issue as a red herring."
Mr Betts said they had had to find a replacement when Alwyn Humphreys, their former musical director, decided to retire after 25 years.
"Sad to say, some people still think Alwyn should be conducting. They're living in the past, they should move on."
Morriston Orpheus has acquired a worldwide reputation, recently returned from a tour of New Zealand and Australia where it performed at the Sydney Opera House.
The 120-stong choir has also branched out into pop and performed with the Welsh band The Alarm and 1980s chart-toppers T'Pau.
"Top quality conductors who can hold together a choir of over 100 men singing four part harmony are not easy to come by," said Mr Betts.
He added it had taken almost 18 months to appoint Ms Pearce and he would be keen to attract her back.