Nicky Butt's stunning strike kept up Graeme Souness' unbeaten start as Newcastle manager.
Butt launched himself into the air and volleyed past Maik Taylor to score his first goal for the Magpies.
Newcastle had gone ahead after three minutes when the unmarked Jermaine Jenas drilled home but Dwight Yorke beat the offside trap to equalise.
Birmingham took the lead when Matthew Upson headed in Savage's superb free-kick before Butt's dramatic reply.
Birmingham boss Steve Bruce might ultimately be happy with a point, given the nightmare start his side suffered.
Inside three minutes, Patrick Kluivert's shot from the edge of the area took a deflection and arrived at the feet of Jenas, who had time to steady himself and shoot into the corner.
And seven minutes later, Birmingham were forced into a substitution, with Emile Heskey limping off to be replaced by Yorke.
But Yorke immediately set about showing his worth to Souness, who he fell out with while both were at Blackburn.
And Dunn, in his first start of the season, also began to trouble the visitors' defence and was only denied a shooting chance after a delightful run into the box when Robbie Elliott intervened with a crucial tackle.
After 15 minutes, a foul on Dunn gave Savage the chance to float a free-kick into the box, which was just too high for Yorke.
And it was no surprise that Dunn had a hand in his side's equaliser.
Yorke timed his run to perfection to stay onside and collect the midfielder's pass before beating keeper Shay Given with a deflected shot.
The Magpies might have retaken the lead just before the half-hour mark when Kluivert saw his header from Craig Bellamy's left-wing cross well-saved by Taylor.
And Birmingham's bad luck continued.
First, Yorke appeared to be pushed by Andy O'Brien as he jumped for a Damien Johnson cross, then Bruce had to make a second substitution when Muzzy Izzet suffered a knee injury.
Still, the hosts might have gone in ahead at half-time had Savage not sent a free-kick just wide of Given's right-hand post.
The Blues continued to press after the break, with Savage again threatening with a free-kick that Given punched away.
But Savage eventually got his reward for a typically-energetic display when another free-kick deceived the entire Newcastle defence, leaving Upson with a simple header.
The goal seemed to spur Newcastle into action but the equaliser came from an unlikely source.
A cross from the left was headed away towards Butt, who somehow managed to contort his body to direct a fierce volley past Taylor.
In a thrilling finale, Birmingham had plenty of chances to take all three points and Newcastle were grateful to Given.
The keeper tipped over after Yorke's turn and shot and then did brilliantly to keep out Dunn's header - as the home fans were already celebrating.
Newcastle had their own opportunities, with Laurent Robert crashing a free-kick against the post and Bellamy appealing in vain for a penalty after he felt he was pushed off the ball by Kenny Cunningham.
Birmingham: Maik Taylor, Melchiot, Cunningham, Upson, Clapham,
Gronkjaer, Izzet (Clemence 36), Savage, Johnson,
Dunn (Gray 83), Heskey (Yorke 9).
Subs Not Used: Bennett, Morrison.
Booked: Dunn, Clemence, Savage.
Goals: Yorke 23, Upson 57.
Newcastle: Given, Carr, O'Brien, Elliott, Bernard (Robert 64),
Bellamy, Butt, Jenas, Bowyer (Hughes 81), Shearer,
Kluivert (Milner 45).
Subs Not Used: Harper, Ambrose.
Booked: Elliott, Butt.
Goals: Jenas 3, Butt 67.
Ref: H Webb (S Yorkshire).