Francois Bizot was the first witness at the trial of the chief jailer, Duch
Talks between Cambodian officials and the UN over funding and alleged corruption at the Khmer Rouge war crimes tribunal have broken down.
Cambodia says it is running out of money to pay staff, but international donors are withholding fresh cash until claims of corruption are addressed.
The deadlock could derail the trials of the surviving Khmer Rouge leaders.
Meanwhile, a French scholar detained for alleged espionage in the 1970s has testified against the chief jailer.
Francois Bizot, who wrote the book The Gate about his experiences, was the first witness to appear at the trial of Kaing Guek Eav, known as Duch, for crimes against humanity.
He said Duch was a man on a mission, and described a "terrifying atmosphere of fear and death" at the main prison, Tuol Sleng, which Duch ran during the 1971-1975 Khmer Rouge insurgency against the then US-backed government.
Last week, Duch apologised for his work at Tuol Sleng, accepting blame for the later extermination of 15,000 people.
As many as two million people are thought to have died during the four years of Khmer Rouge government in the late 1970s.
Duch is the first of the surviving Khmer Rouge leaders to go on trial at the UN-backed court - but the UN-Cambodia wrangling is threatening to derail the process.
Following three days of talks, a UN spokesman said the two sides had failed to reach a deal.
WHO WERE THE KHMER ROUGE?
Maoist regime that ruled Cambodia from 1975-1979
Founded and led by Pol Pot, who died in 1998
Abolished religion, schools and currency in a bid to create agrarian utopia
Up to two million people thought to have died from starvation, overwork or execution
"We were very close to an agreement and I have left a proposal on the table of deputy premier Mr Sok An and asked him to consider it," said the UN assistant secretary general for legal affairs, Peter Taksoe-Jensen.
The tribunal is a Cambodian court - but UN-appointed judges and legal officials play key roles, and almost all of its running costs are met by international donors.
The administration is split into local and international sides - and it is the Cambodian half which has run into trouble, saying it is almost out of money for staff salaries.
But allegations that local staff have been forced to pay bribes for their jobs have led many donors to suspend funds until methods are in place to ensure cash is not siphoned off.
The visiting UN delegation insisted that local staff should be able to report corruption to an international "ethics monitor" - especially if they feared retaliation.
Lawyers and human rights groups say that the allegations of corruption could sink the court's credibility if not resolved.