A Russian general said his forces were there to remove weaponry and help restore law and order in Gori, which lies some 15km (10 miles) from South Ossetia and on a key route to the Georgian capital, Tbilisi.
The BBC's Natalia Antelava in Tbilisi said plans for a joint patrol force by the Georgian police and Russian military had failed.
Our correspondent said there were also reports of Russian military vehicles moving around the town of Senaki and the Georgian Black Sea port of Poti in western Georgia.
Russia has questioned what is in US aid deliveries to Georgia
Moscow had earlier denied the reports but Russia's deputy chief of staff, Gen Anatoly Nogovitsyn, told a televised news conference it was legitimate for Russians to be in Poti as part of intelligence-gathering operations.
Georgia's Prime Minister Lado Gurgenidze said a convoy of more than 100 Russian tanks and other vehicles was moving from the major western town of Zugdidi deeper into Georgia, but officials were later quoted as saying the column had turned back.
Mr Gates said that despite concerns that Moscow may not be keen quickly to leave Georgian territory, the Russians did seem to be pulling back.
"They appear to be withdrawing their forces back towards Abkhazia and to the zone of conflict... towards South Ossetia," he said.
Gen James Cartwright, vice-chairman of the US Joint Chiefs of Staff, also said he believed Russia was "generally complying" with the terms of the truce, which called for its withdrawal from hostilities.
But, Mr Gates warned: "If Russia does not step back from its aggressive posture and actions in Georgia, the US-Russian relationship could be adversely affected for years to come."
The Russians were trying to redress what they regarded as the many concessions forced on them after the break-up of the Soviet Union and were trying to "reassert their international status", Mr Gates said.
Georgia was also being punished for its efforts to integrate with the West and in particular to join Nato, the defence secretary went on.
The BBC's Justin Webb in Washington says Mr Gates's address was the first effort by a senior member of the Bush administration to set out what the Americans believe is happening in Russia.
But while Mr Gates said Russia's aggressive posture was not acceptable, our correspondent says, he took an unusual step for the Bush administration in ruling out the use of US force. This is not a fight that America wants to have.
UN Secretary General Ban Ki-moon has said he is "extremely concerned by the humanitarian impact" of the conflict on the civilian population. He warned that aid agencies were not able to reach areas worst-affected by the conflict "due to ongoing insecurity, lawlessness and other constraints".
He said "all fighting should end immediately and the current state of lawlessness should cease".
Georgia attacked the rebel region of South Ossetia from Gori a week ago, prompting Russian retaliation. The Georgians say it followed continuous provocation.
A Georgian state TV reporter was injured by gunfire while she was on air
Both sides agreed to a French-brokered ceasefire on Tuesday, amid international concern, but it has seemed fragile so far.
Earlier on Thursday in Moscow, Russia's President Dmitry Medvedev said Russia would respect any decision South Ossetia and Abkhazia made about their future status.
His words followed warnings from the US that Russia had to respect Georgia's territorial sovereignty and withdraw its forces.
Meanwhile, the US has sent its second shipment of humanitarian aid into Georgia.
Russia has questioned whether the deliveries contain only humanitarian supplies.
CNNShaky Cease-fire between Russia and Georgia; British Television Reporter Roughed Him Up During a Pro-Tibet Rally; Trend in Foreclosures Continues Increasing; Comedian Margaret Cho Returns to TV - 18 hrs ago
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.