The museum traces Jewish history since Roman times
More than 800 guests attended the gala opening ceremony, including the German Chancellor Gerhard Schroeder, President Johannes Rau and former US Secretary of State Henry Kissinger.
The museum, which was completed two years ago, has already established itself as one of Berlin's most visited attractions because of its extraordinary architecture.
The exhibition, which opens to the public on Tuesday, traces the history of the Jews in Germany from Roman times through to the revival of Jewish life today.
The extraordinary architecture of the zinc-clad museum, designed by the Polish-born architect Daniel Libeskind, starkly reminds people of the Holocaust.
The building - with its windows of irregular shards of glass - has been likened to a streak of lightning, and has become one of Berlin's main tourist magnets.
Emotionally overwhelming
The centre point of the museum is the Holocaust Tower - a door slams shut behind visitors, who find themselves in a barely lit concrete chamber.
The BBC's Berlin correspondent Peter Morgan says it is an eerie place, representing the void left by the millions of Jews who died in the Nazi extermination camps.
Mr Libeskind lost much of his own family in the Holocaust.
For many it is emotionally overwhelming - tours of the empty building provoked strong reactions. Some people found the museum so disturbing they had to leave.
Cultural contribution
But the museum's designers are anxious that the massive cultural contribution of the Jewish community is not entirely overshadowed by the unspeakable horror of the Nazi regime.
Our correspondent says it was fitting therefore that the museum opening began with a symphony by Mahler, a Jew who lived in Leipzig and in Hamburg.
His work was conducted by Daniel Baramboim, who is also Jewish and is the musical director of one of Germany's great opera houses.
Religious artefacts
The new museum springs from a decision by the post-war Berlin city to expand its Jewish department. In 1989 it chose Mr Libeskind's design for a new building.
The objects on display will include books, religious artefacts such as Torahs, historic documents, objects from daily Jewish life - such as instruments used to circumcise Jewish boys - and even the glasses of an 18th century Jewish philosopher, Moses Mendelssohn.
The Nazis closed the city's previous Jewish museum in 1938.