Stirling Mortlock's first-half try and brilliant goalkicking from Elton Flatley earned the hosts victory.
New Zealand had been expected to win the match, but they couldn't break through an excellent Australia defence and failed to kick their penalties.
Australia now face either England or France - who play their semi final on Sunday - in the final on 22 November.
The All Blacks speedy backs were thought to be too much for the hosts, but Australia kept the ball away from them.
And when New Zealand did get the ball they made too many mistakes, including the interception that let Mortlock score his try.
Hear story
Vote: Who will win the Rugby World Cup?
Rugby World Cup quiz
