Over 100 firefighters were called to the blaze at the 53-year-old Lido
Damage to the Afan Lido leisure centre caused by a severe fire in December is so bad that the majority of it will have to be demolished.
Neath Port Talbot council said the swimming pool, reception area and ballroom were structurally unsound and beyond reasonable repair.
The sports hall may yet survive but can not be used until the rest of the site is made safe.
A planned gig by the Lostprophets in March is now unlikely to go ahead.
The fire began at the poolside on 16 December and at its height had over 100 firefighters at the scene trying to extinguish the flames.
Following the damage, the council pledged to either repair or replace the 53-year-old building, which included the Aquadome water slides and tyre rides facility, a swimming pool, spar and a music venue.
Steve Phillips, chief executive of Neath Port Talbot, said: "A structural assessment has been carried out on the Afan Lido complex by independent structural engineers.
"This has confirmed very severe damage has resulted from the fire to the building fabric and primary structure of the pool hall.
"The report advises that extensive demolition is necessary and officers will be recommending a way forward to the council once financial and technical appraisals for demolition have been completed.
"Air tests carried out on the night of the fire, and subsequently, show normal air quality readings, confirming that asbestos disturbed as a result of the fire is confined inside the building.
"The council will be starting the process of considering the options for the provision of leisure facilities on the seafront and will be consulting with the public later this year on its proposals."
The council's decision has cast major doubt on the venue's ability to host the planned St David's Day gig by Pontypridd rockers the Lostprophets.
The band release their fourth studio album, Betrayed, on Monday, and as recently as Saturday the band's record company, Visible Noise, told the South Wales Evening Post that the Afan Lido date would be going ahead despite the fire.
However, a spokesman for Neath-Port Talbot Council told BBC Wales: "It's too early to say that the gig definitely couldn't take place in the Afan Lido sports hall, but given the time frame, the estate managers there have recommended that they try to find an alternative venue.
"Whilst the hall itself is undamaged, the mains electricity supply comes from the pool area, and the exits require people to cross the cordoned-off zone."
Visible Noise say the situation was less clear than they had been led to believe at the end of last week, but reassured fans that if the Lido wasn't ready in time, all tickets would be refunded or honoured for an alternative date.