Home Office inspectors have criticised Leicester Prison after nine inmates died there in a 28-month period.
Nine inmates died at the prison during 28 months
The Chief Inspector of Prisons Anne Owers said the deaths were "of particular concern".
Her report said seven of the inmates had apparently taken their own lives. Overcrowding and poor health at the prison were also highlighted.
Prison governor Danny Mulligan said work had been done to make first-night cells safer for new inmates.
The doors and windows had been replaced so that inmates could not fix a ligature to them and hang themselves, he said.
"I'm disappointed with the report, certainly for staff because the report does actually recognise the good relationships between staff and prisoners," he added.
"But deaths in custody is something very close to my heart and I'm determined to do everything I can to reduce the risk.
"We've replaced the windows with anti-ligature windows and the furniture with anti-ligature furniture.
"That programme needs to be developed further, I recognise that, and I've actually managed to secure funding so that by the end of the financial year all of the cells in our first night centre will have been refurbished."
Derek Goodman, who heads the prison's self harming committee, said: "The committee has been working hard to improve relations with prisoners and making sure that self harm does not occur.
"I remember in one month four years ago there were 50 incidents of self harm, in June this year there were five, that's a measure of what's been happening."
Ms Owers said Leicester Prison's problems were characteristic of a Victorian, inner-city jail.
"Leicester shows, in a microcosm, some of the problems faced by an overcrowded and stretched prison system," she said.
"It is not essentially a prison with a negative culture. Nevertheless, some of the deficits, particularly in safety, are unacceptable and need urgently to be remedied."
On Tuesday there were 347 inmates in the 205 cells in Leicester Prison.