The US and UK have pledged to back Iraq's new government and pursue militants responsible for violence.
Bremer (right) said he was confident in the future of Iraq
A new American ambassador to Iraq flew in shortly after the US administrator, Paul Bremer, handed over power to the Iraqis and left the country on Monday.
The State Department said John Negroponte would not be a new administrator but a true ambassador.
An Arabic TV station has broadcast a video which it says shows a US soldier killed by his militant captors.
He was shot dead, al-Jazeera said, but the tape did not show the actual killing.
The station named him as Private Keith Maupin, who was one of a group of Americans seized in April, west of Baghdad.
Apart from a video sent to al-Jazeera a week after his capture, he had not been heard of since then.
Several explosions were heard in Baghdad late on Monday night - some reportedly close to the US headquarters in the so-called Green Zone - but there were no reports of casualties.
The handover of power had been scheduled for Wednesday, but took place two days early, with little prior warning, apparently because of fears of possible attacks.
"The Iraqi people have their country back" as promised, Mr Bush said outside an Istanbul summit of 26 Nato countries, which approved a plan to train Iraqi security forces.
'Let Freedom reign!' - Mr Bush responds to historic handover note
He said the security situation in Iraq was "tough".
Iraqi interim Prime Minister Ayad Allawi's government had to take strong measures against al-Qaeda suspect Abu Musab al-Zarqawi - described by President Bush as a "brutal cold-blooded killer" - who is said to be behind many anti-coalition attacks.
Asked whether he would allow US troops to assist Iraqis in implementing martial law if the prime minister introduced it, Mr Bush said Iraq had a sovereign government and US troops would train Iraqi forces to go after Zarqawi.
British Prime Minister Tony Blair said the new Iraqi government would abide by human rights in taking tough measures - it would not seek to curb liberties.
"Al-Qaeda and other terrorist groups are actually there in Iraq now, what is happening in Iraq, the battle in Iraq, the battle for Iraq and its future, if you like, is, in a genuine sense, the front line of the battle against terrorism and the new security threat that we face."
Kuwait has announced that it is reestablishing diplomatic relations with Iraq, 14 years after Saddam Hussein invaded the country.
US administrator Paul Bremer handed over power at a low-key ceremony inside the heavily-guarded Green Zone on Monday before flying out of the country.
Mr Bush and Mr Blair - apparently the only leaders at the Nato summit aware that the handover was taking place - exchanged smiles and a brief handshake after consulting their watches.
Iraq's interim prime minister said it was "a historic day".
In a more colourful ceremony hours later, Mr Allawi and his Cabinet ministers were officially sworn in. But even this oath-taking was held in secret.
"I call on our people to stand united to expel the foreign terrorists who are killing our children and destroying our country," Mr Allawi said in a televised address.
The transfer was widely welcomed - the European Union and the Nato alliance both pledged their support for Mr Allawi's government.
The BBC's world affairs editor John Simpson in Baghdad says bringing the handover forward was a clever tactical move that wrong-footed the Iraqi resistance.
However, he says it actually underlines the US-led coalition's big failure - its inability to stop the violence.