A perimeter fence surrounding the home of Stagecoach tycoon Ann Gloag is vital for security, her husband has claimed.
Security at the castle had become more important, the court heard
Businessman David McCleary, 56, told Perth Sheriff Court that there had been problems with people taking photos from the gates of Kinfauns Castle.
Mrs Gloag, who is one of the richest women in Scotland, has launched a legal bid to ban the public from part of her estate in Perthshire.
She is seeking an exemption from right-to-roam legislation.
Mr McCleary also said he was informed by police that a known criminal was plotting to rob Mrs Gloag, although the plan was "nipped in the bud" at an early stage.
A perimeter fence erected around part of the 23-acre estate without planning permission was later granted retrospectively by Perth and Kinross Council.
The local authority and the Ramblers Association said they wanted a stretch of the one-mile fence moved.
Mr McCleary said having the measure was now more important in light of recent trouble.
He said: "There are people with cameras. I think because they are going up to the gates of a baronial mansion they may think it's a film star living there."
Mr McCleary added that it would be "madness" to live in the castle without a perimeter fence.
"Unless I had a gun you wouldn't want to be there," he said.
"I have had to call the police three times in the last six weeks."
Perth and Kinross Council, backed by the Ramblers Association Scotland, are seeking to move the boundary fence closer to the castle.
The court hearing, before Sheriff Michael Fletcher, continues.