The governor of North Sea Camp open prison has been suspended pending an investigation, the BBC has learned.
Inspectors carried out an unannounced visit on Friday
Keith Beaumont was reportedly escorted off the premises at Freiston near Boston, Lincolnshire, on Friday.
The Home Office has confirmed that an investigation has begun into allegations of misconduct.
The Prison Service has appointed two acting governors to take over the running of North Sea Camp during the inquiry.
Allan Clark, the deputy governor at HMP Sudbury, in Derbyshire, and Norman Warwick, acting governor at HMP Morton Hall in Lincoln will take over the job.
BBC Radio Lincolnshire has learned that the suspension happened after an unannounced visit by inspectors who carried out a thorough search.
A Prison Service spokesman spokesman said an investigation into alleged misconduct at the jail will start on Sunday.
He said: "We suspended the governor... pending a full investigation. The allegations relate to unprofessional conduct."
Lord Archer was a famous inmate
The suspension is the latest controversy at the camp.
Claims that drugs are freely available at the prison have been made in a string of court cases involving prisoners who have absconded from the camp.
One prisoner walked out in protest at conditions in the prison, and later told Lincoln Crown Court he was worried he might start taking drugs again if he stayed.
The open prison - which houses just over 300 inmates - was subjected to national publicity in 2002 when it housed Lord Archer, who was serving a sentence for perjury and perverting the course of justice.
Lord Archer was moved out of the prison after he broke the rules by attending a party during a day release.
A previous unannounced visit by the Prisons Inspectorate in July 2002 found that only 19 recommendations from an inspection had been implemented, but 35 had not been achieved at all.
"Some basic standards of prison and prisoner management...were undeveloped or lacking," chief inspector of prisons Anne Owers said in the report.