By Guy De Launey
BBC News, Phnom Penh
Mr Ghai has spent a week in the country
The new UN human rights envoy to Cambodia has said it is time to worry about the situation in the country.
In recent weeks, two government critics have been arrested and jailed, and several others fled the country after they were named in arrest warrants.
The EU has formally asked the government to release the two men in jail as well as an opposition MP jailed for attempting to raise an army.
Although he is from Kenya, UN envoy Yash Ghai is no stranger to Cambodia.
He was involved with drafting the Cambodian constitution 12 years ago.
Now he is concerned the rights enshrined in that document are being eroded. He says there are two opposing groups in Cambodia - those who think human rights are the cause of all problems, and those who think they are the only answer.
The envoy believes this is causing deep divisions in Cambodian society.
He says there has been a steady attrition in human rights protection and democratic practise which has come to a boil in recent weeks. It is a trend he would like to reverse:
"The government here seems to think that human rights are a nuisance. I am hoping that in my time here, I can persuade the government of the value of human rights. For the government itself, it gives them legitimacy, values - instruments with which to bring people together.
During his visit Yash Ghai met the three opposition figures who are currently in prison.
He believes that all of them should be released, and that the criminal defamation charges against two of the men are highly suspect and unconstitutional.
The European Parliament has passed a resolution calling for a delegation to visit Cambodia to examine the human rights situation.
A Cambodian government spokesman said they would be welcome as long as they did not interfere in the country's efforts to fight terrorism.