A report has praised recent improvements made at Leicester Prison but said it was still "dirty and unwelcoming to prisoners".
Two and a half years ago inspectors described the prison as 'failing'.
The new report says progress has been made but more work needs to be done.
One area for improvement is that some prisoners at risk of self harm are not being watched correctly.
Race relations and anti-bullying strategies were also found to be out of date.
Steve Turner who has been governor of Leicester Prison for three months and came into the post after the inspection was carried out, said: "I accept the criticisms that have been made and I'm also pleased to see that good practice was recognised."
The report described the jail as "failing" in February 2001 and in August of this year it was the second most overcrowded in the country.
One prisoner disagreed and told the BBC: "The support is good, they are willing to help you.
"Even when you are ready to leave they help set up things for when you get out there."
Mr Turner went on to say: "I think Leicester should be addressed as an improving prison.
"We have targets that have been set and I am confident with the staff behind me that we can quickly address those targets."