The plant currently buys cheaper electricity from Wylfa power station
Compulsory redundancy notices for 300-350 workers at Anglesey Aluminium are set to go out later this week, according to union Unite.
The union says eventually it could lead only 81 workers remaining at Holyhead, melting down metal, not producing it.
The union says no headway has been made since the firm said a £48m UK government aid package was too little.
A company spokesperson said management would this week be looking at "what the next steps should be".
"The next plan of attack is that [Welsh Secretary] Peter Hain's office is still working with the Tinto (major shareholder) office to try and get more discussion," said Graham Rogers, of Unite.
"Hopefully we can then talk to members of their board and Anglesey Aluminium board."
Mr Rogers said he and two others had travelled back from London last week "feeling like we'd won the lottery", due to the rescue plan offer - only to be disappointed when the company said "thanks, but no thanks".
He said it was sad the company was not now willing to back the site until a bio-mass plant could be built, so that the works was self-sufficient in energy.
He said he would never give up, but it seemed the gap between the aid offer and what the company wanted was "too much".
"It is very sad, because the plant has been there over 40 years," Mr Rogers added.
Last week the firm rejected UK government aid of £48m over four years because it was "not enough to break even".
A community action group met on Friday to discuss the impact of potential closure of the plant which currently employs 500 people.
The group, which includes politicians and company management, said it had been "a very constructive meeting".