Staff at Wakefield Prison, which holds some of Britain's most dangerous men, have been criticised for being "disrespectful" to inmates.
Inspectors found "an atmosphere of disengagement"
The West Yorkshire prison was "over-controlled" and a third of inmates claimed to have been victimised, the Chief Inspector of Prisons said.
Some claimed to have been assaulted by "bullying" officers, Ann Owers said.
Governor John Slater dismissed assault allegations and said he was surprised by some of the inspection findings.
The percentage of prisoners claiming to have been victimised rose to 54% for the over-60s, inspectors found after the unannounced inspection in October.
They said that, instead of handcuffing all prisoners during unscheduled moves, individual risk assessments should be carried out.
Britain's worst serial killer, Harold Shipman, was found hanged in his cell at Wakefield on 13 January and other notorious inmates at the jail include Roy Whiting, who was convicted of murdering Sarah Payne.
Its close supervision centre holds some of Britain's most dangerous men, including serial hostage taker Charles Bronson and Robert Maudsley, jailed in 1974 for garrotting an uncle.
Maudsley has since killed three fellow inmates and was suspected of eating part of the brain of one.
Miss Owers said: "We saw one incident where a prisoner on crutches was handcuffed and the handcuffs jerked by the accompanying officer.
"We were concerned that prisoners in the [segregation] unit consistently referred to a climate of bullying and intimidation there. There were even a few allegations of assault."
But she said prisoners were reluctant to make formal complaints "for fear of making things worse".
Speaking ahead of journalists being allowed in Wakefield prison for the first time on Thursday, Mr Slater said there was "absolutely no substance" to the assault allegations.
He also intimated that inspectors may not have fully appreciated the security requirements of a prison with such dangerous offenders.
Inspectors said inmates had dubbed officers' detached style "the Wakefield way" and that inspectors observed a lack of interaction between inmates and staff which, Miss Owers insisted, meant officers received "relatively little security intelligence" from the prisoners themselves.
Inspectors also found there was an "extremely high" number of complaints from inmates about bullying but said procedures to deal with complaints were thorough.
Miss Owers concluded there was an "atmosphere of disengagement amounting to disrespect".
Director of the Prison Reform Trust Juliet Lyon said: "Indications of maltreatment and bullying by prison officers revealed in this report must be taken extremely seriously and acted upon immediately by the Prison Service.
"People are sent to prison to be deprived of their liberty not for additional punishment and, whatever crimes they have committed, the Prison Service has a duty to care for prisoners and treat them with respect and decency at all times."
Wakefield prison will become Europe's largest high-security prison when a new wing opens next year.
The jail has 570 inmates, with 70% serving life sentences.
At the time of the inspection in October, there were 193 men convicted of murder and 275 sex offenders.