The former leader of the council, who oversaw the building of Swansea Leisure Centre 27 years ago, says he has come out of retirement to fight its closure.
The site's permanent closure was announced in November
Gerald Murphy is heading the campaign to save the centre - which is in need of £14m-worth of repairs.
Protesters attended a public meeting at Swansea's Greyhound Stadium, in Fforestfach, on Sunday to debate the future of the site.
Swansea Council want to see a multi-purpose arts and sports venue, up to eight storeys high, replace the centre.
The permanent closure of the building was announced in November but Mr Murphy said more should be done to keep it open.
"We want the council to have another look because we know that it is well used," said Mr Murphy.
"To destroy that facility is ridiculous, it is one of the most popular centres in Wales.
"If it wasn't being used I would agree with them but it is.
"The people feel that once the centre is closed that is it, it is finished and you will never have another leisure centre like it.
"I was the leader of the council when the centre was built.
"It was paid for by the people of Swansea and they should have a say on what happens to it now."
The man behind the spectacular Guggenheim museum in Bilbao has been tipped as one of the contenders to design a replacement for the centre.
Swansea council are seeking a private partner for a £35m development which would include a conference centre and concert hall.
Robert Francis-Davies, the council's cabinet member for culture and recreation, said there is a possibility world-famous architect Frank Gehry may come to work on the project.
The council are taking advice from leading property designers CB Richard Ellis who are working with Frank Gehry on a similar project for Brighton and Hove Council.
Eight companies have approached Swansea council with proposals for the site but they have not yet entered into any negotiations.