The charity campaign group Save Britain's Heritage has called for Newark Works in Bath to be spot listed and plans for its demolition shelved.
The factory building is to be replaced with a city academy
The former factory on Lower Bristol Road is to be replaced with a city academy sponsored by James Dyson.
The 18th Century Bath stone building was created by architect Thomas Fuller who is responsible for the acclaimed Ottawa Parliament in Canada.
Adam Wilkinson, of SAVE, said: "This is Fuller's best work in the UK."
The Newark Works consists of a grand classical entrance block, three bays wide and three storeys high in Bath stone with details picked out in darker stone. To either side are wings with large windows.
"It is shameful that is threatened with demolition," said Mr Wilkinson, secretary of SAVE.
"Its demolition is entirely unjustified - if this was in North America it would almost certainly be protected."