The two sides have had troops and equipment in the area since July
Thai soldiers have been accused of crossing into Cambodia near a disputed temple where the two sides briefly exchanged fire last year.
A spokesman for Cambodia's government said that about 100 troops crossed the border before retreating hours later.
A Thai border commander denied there had been any troop movements and said there had been no increase in tension.
Thailand and Cambodia both lay claim to the temple area. Despite several rounds of talks, a settlement remains elusive.
Soldiers from the two countries have been stationed in the area since the clashes in July last year.
"Now, there is no more confrontation between the two sides," Cambodia Brig Gen Yim Pim was quoted as saying by the Associated Press.
"We asked them to pull back their forces, and they agreed."
Earlier, government spokesman Phay Siphan told reporters that Thai troops had gone about a kilometre into Cambodian territory.
"We are negotiating with their commanders to ask them to leave the area now because it is Cambodian territory," he said at the time.
Col Pichit Nakarun, a Thai army commander at the border, denied any Thai troops had crossed the border.
"The situation is not more tense than usual," he told AP.
An international court awarded the Preah Vihear temple to Cambodia in 1962 but land surrounding it remains the subject of rival territorial claims.
Last year Unesco decided to list it as a World Heritage Site, reigniting lingering tensions over unresolved border disputes.
Thai troops moved into an area both sides claim after Cambodian guards arrested three Thai protesters there.
Both sides then rapidly increased their military presence at the site, and in October two Cambodian soldiers were killed in an exchange of gunfire.