Prison officers at Dartmoor jail in Devon have passed a vote of no confidence in the governor.
Dartmoor has about 640 prisoners
Staff are unhappy about the way Serena Watts is running the category C jail and communicating with the workforce.
Trevor Horn, of the local Prison Officers Association (POA), said staff felt bullied by managers and unsafe.
No-one was available for comment from the prison, which government inspectors said in 2006 had "improved beyond recognition".
The vote came at a meeting of the POA attended by about 55 of the union's 200 members at the prison.
Category C prisoners are those who cannot be trusted in open conditions but who are unlikely to try to escape.
Mr Horn said the action by staff would not affect security at the prison, which houses about 640 prisoners.
The only tangible effect of the vote is that staff will only communicate with the governor in writing.
He said three main factors had influenced the vote - "bullying and harassment" of staff by the senior management team, the loss of an Investors in People award and the escape of a prisoner in 2006 from Derriford Hospital in Plymouth.
He said: "All prisons have their ups and downs, but staff feel on their own and they are not getting the leadership they expect."
He also said staff felt "lonely and unsafe" on the wings after a drop in visits by managers.
The Ministry of Justice said: "This is a matter for the Prison Officers Association."
Seven years ago the prison was condemned by HM Inspectorate of Prisons as "the prison that time forgot" and was threatened with privatisation unless it improved its performance.
Since then, however, it has won a number of awards including the public service team of 2004-05 title, the prison officer of the year 2004-05 title and the most improved prison award in 2005.
A 2006 report by the government inspectors said it had been "transformed" and was now fit for the 21st Century.