Ivory Coast came from two goals behind to beat 10-man Serbia & Montenegro.
Serbia were 2-0 up after 20 minutes, Nikola Zigic rounding Boubacar Barry for the first and Sasa Ilic punishing Cyril Domoraud's poor control.
Aruna Dindane then scored a twice-taken penalty before Albert Nadj was sent off at the end of the first-half.
Dindane's header made it 2-2, before Bonaventure Kalou scored a second penalty, though Ivory Coast ended up with 10 men after Domoraud's dismissal.
The win was Ivory Coast's first in the World Cup finals, delighting the 66,000 crowd at Munich's Allianz Arena.
It was also the first time a team had come back from two goals down to win a match in the competition since West Germany's epic semi-final win over France in 1982.
The win might have been historic, but neither team could be said to have bowed out of the World Cup in style, given some comical defending and the spate of yellow and red cards.
Nadj had only been on the pitch as a first-half substitute for 30 minutes before his dismissal after he picked up two yellow cards.
Serbia exploited some woeful defending from the African side as they took control of the match.
606 VIEW
After stringing a series of passes together, Dejan Stankovic's diagonal pass was deftly controlled by Zigic, who took the ball round the stranded Barry.
Both Barry and central defender Blaise Kouassi were poorly positioned for Zigic's goal but their mistakes were nothing compared to Domoraud's slip for Serbia's second.
The Creteil defender completely failed to deal with Predrag Djordjevic's free-kick and Ilic pounced to guide the ball home.
The comedy of errors continued when Milan Dudic bizarrely raised his hand after Arthur Boka crossed from the left.
Dindane had to twice take the penalty - his first effort was disallowed for encroachment by Ivorian Arouna Kone - but the Lens striker stayed calm to wrongfoot Serbia goalkeeper Drajoslav Jevric.
If only Dindane had showed similar composure when chances came to him in open play.
Just before the interval Jevric was lucky not to concede a penalty when he appeared to fell Dindane.
Early in the second half Jevric made a good save with his legs, diverting Kanga Akale's shot to safety after Boka's free-kick was blocked.
Soon after Kouassi hit a vicious shot that Jevric could only watch as it skimmed the bar.
Midway through the second half Dindane's persistent forward play was rewarded when he powered a header past Jevric.
Dudic gifted the Ivorians a second penalty with another handball and substitute Kalou made no mistake from the spot.
In stoppage time Mexican referee Marco Rodriguez brandished one final card - a red to Doromaud after he was booked for a mistimed tackle on Savo Milosevic.
Ivory Coast: Barry, Boka, Kouassi, Domoraud, Eboue, Akale (Bakari Kone 60), Zokora, Keita (Kalou 72), Gneri Yaya Toure, Arouna Kone, Dindane.
Subs Not Used: Demel, Fae, Gnanhouan, Meite, Romaric, Tizie, Kolo Toure, Yapi Yapo, Zoro.
Sent Off: Domoraud (90).
Booked: Keita, Domoraud, Dindane.
Goals: Dindane 37 pen, 67, Kalou 86 pen.
Serbia & Montenegro: Jevric, Gavrancic, Nenad Djordjevic, Dudic, Krstajic (Nadj 16), Ergic, Duljaj, Stankovic, Predrag Djordjevic, Ilic, Zigic (Milosevic 66).
Subs Not Used: Basta, Dragutinovic, Kovacevic, Ljuboja, Stojkovic, Vidic, Vukic, Petkovic.
Sent Off: Nadj (45).
Booked: Nadj, Dudic, Duljaj, Gavrancic.
Goals: Zigic 10, Ilic 20.
Att: 66,000.
Ref: Marco Rodriguez (Mexico).
Fifa man of the match: Aruna Dindane
TRIVIA
Ivory Coast beat Serbia & Montenegro 3-2 after being two goals down. The Africans are only the eighth team to overcome a two-goal deficit at a World Cup.
Albert Nadj (Serbia & Montenegro) became the 15th substitute to be red carded at the World Cup. Nadj is the first sub to see red in the first half. Both teams received a red card. It was the 16th World Cup encounter in which both sides had a man sent off.
Serbia & Montenegro netted their first goals of the tournament, in this 3-2 defeat. That equals the least number of goals scored by either Serbia & Montenegro or Yugoslavia in a single World Cup finals. The other lowest returns were in 1954 and 1982.
Trivia stats source: Infostrada Sports