Up to 150 prisoners are being moved into a prison wing which was closed at the weekend after being condemned as unfit for human habitation.
The Home Office has told Norwich Prison to take in remand prisoners who are currently being held in police cells.
The Independent Monitoring Board (IMB) has criticised the decision.
IMB member Trish Phillips said: "It's horrible, an old Victorian block in a very bad state of repair and difficult to decorate."
Ms Phillips told the BBC: "About 20 years ago when it was decided that it was really not acceptable to have buckets in cells, the prison put in soil stacks and integral toilets but it was done on the cheap and the soil stacks leak."
The IMB is responsible for monitoring the welfare of prisoners.
James Shanley, the prison's governor, said this was a national issue for which the Home Office was responsible and he was not allowed to comment.
A Home Office spokeswoman said 150 places would be brought back into short-term use to house prisoners on remand from the courts.
Cells that are not suitable for occupation will not be certified for use.