Oliver Neuville put Germany within reach of the last 16 with a goal in the first minute of stoppage time to leave Poland pointless after two games.
The forward latched on to a low cross from fellow substitute David Odonkor to slide the ball past the despairing dive of Poland goalkeeper Artur Boruc.
The goal came a minute after Miroslav Klose and Michael Ballack had hit the bar within seconds of each other.
Poland's Radoslaw Sobolewski was sent off late on for two bookable offences.
The Germans created a hatful of chances, but failed to make the most of them and also found Boruc in sensational form.
Klose should have eased German nerves earlier in the game, but the 28-year-old could not add to his two-goal tally from the first game against Costa Rica.
The Werder Bremen striker, who scored 25 goals in the Bundesliga last season, came close twice in the first 15 minutes with a header and low shot that the Poland keeper smothered.
He then failed to hit the target with another header and struck the crossbar in the second half with a similar effort.
His strike-partner Lukas Podolski also came close to scoring, but his shot from Philipp Lahm's low cross went wide of Boruc's left-hand post.
At the other end, Arsenal goalkeeper Jens Lehmann had little to do. The Poles threatened on several occasions, particularly in the second half with Ireneusz Jelen pulling the strings in midfield.
But for all their good intentions and vibrant build-up play, the final ball let them down.
When they did get a sight on goal, it was Jelen who came closest to scoring, but his second-half strike from the edge of the area went straight down Lehmann's throat.
Germany brought on the pacy David Odonkor in the last 20 minutes and his arrival appeared to add a spring into the step of his frustrated side.
However, the Poland keeper was in no mood to lie down and made fantastic reaction saves from the industrious Lahm and Neuville.
When he was beaten the woodwork saved him as in the last minute of normal time Klose headed against the woodwork from close range and Ballack did the same with the rebound.
But with just a few seconds of stoppage time remaining, all of Boruc's good work was undone by the instinctive reactions of Neuville.
Odonkor sped down the right to fire in a low ball which found the diminutive forward who eluded his two markers to prod past the helpless Poland shotstopper.
Neuville's goal gave Germany their first win against another European team at the finals of either the World Cup or European Championships since their victory over the Czech Republic in the final of Euro 1996.
Germany: Lehmann, Friedrich (Odonkor 64), Mertesacker, Metzelder, Lahm, Schweinsteiger (Borowski 76), Ballack, Frings, Schneider, Podolski (Neuville 71), Klose.
Subs Not Used: Jansen, Huth, Kehl, Nowotny, Kahn, Asamoah, Hitzlsperger, Hildebrand.
Booked: Ballack, Odonkor, Metzelder.
Goals: Neuville 90.
Poland: Boruc, Bosacki, Baszczynski, Bak, Zewlakow (Dudka 83), Sobolewski, Krzynowek (Lewandowski 77), Smolarek, Radomski, Zurawski, Jelen (Brozek 90).
Subs Not Used: Jop, Gancarczyk, Kosowski, Szymkowiak, Rasiak, Kuszczak, Mila, Giza, Fabianski.
Sent Off: Sobolewski (75).
Booked: Krzynowek, Sobolewski, Boruc.
Att: 65,000
Ref: Luis Medina Cantalejo (Spain).
Fifa man of the match: Philipp Lahm
TRIVIA
This was the first ever World Cup victory for (West) Germany through an injury-time goal. Three times before, Germany have won a World Cup match with the winning goal being scored in the last five minutes. The last time this happened, Oliver Neuville was also the scorer. On 15 June 2002, Neuville's 88th minute goal gave Germany a 1-0 victory over Paraguay in the second round.
This narrow triumph in Dortmund means that Germany remain unbeaten against Poland, winning 11 and drawing four, and now have a record of 14 wins and one draw in international matches played in Dortmund.
The red card shown to Radoslaw Sobolewski was the fourth of the 2006 World Cup.