The tribunal, which has been ratified by more than 70 countries, will have power to try individuals for crimes committed anywhere in the world.
But the US has withdrawn from the treaty which establishes the court, and many important states - including most Arab states, Israel, China and Russia - have failed to sign or ratify it, causing some to question the court's credibility.
Hours ahead of the court's opening, the US vetoed the renewal of the UN mission in Bosnia over concerns that its peacekeepers could be prosecuted by the ICC.
Washington says it fears its troops could become the target of politically motivated prosecutions.
But experts say America's concerns that the court will infringe US sovereignty are groundless, because it will only deal with cases that domestic courts are unable or unwilling to handle.
First cases
The UN Secretary General, Kofi Annan described the creation of the court as an "historic moment" and called on all states that have not yet ratified the ICC's statute to do so as soon as possible.
"It holds the promise of a world in which the perpetrators of genocide, crimes against humanity and war crimes are prosecuted when individual states are unable or unwilling to bring them to justice," he said in a statement.
The BBC's Geraldine Coughlan in The Hague says the first cases relating to any of around 40 current conflicts worldwide are expected to be filed at the ICC's temporary headquarters almost immediately.
It will then be up to the prosecutor, who is to be appointed early next year, to decide which cases are politically motivated and which ones are genuine allegations of violations of human rights.
Deterrent effect
The ICC does not have retroactive jurisdiction and can only try crimes committed after 1 July:
The tribunal is expected to be fully operational by the end of next year.
Human rights campaigners hope it will deter future dictators, their officials and armies from committing atrocities because they know there is a forum where they can be brought to account.
But some analysts have warned there is a danger that the ICC will be seen as geographically unrepresentative and Western-dominated.