Ukraine reached the quarter-finals after beating Switzerland on penalties.
Gusev celebrates scoring the decisive penalty
The Swiss missed three spotkicks, but Ukraine's Artem Milevskiy, Serghiy Rebrov and Oleg Gusev made no mistake to seal a 3-0 shoot-out victory.
During normal time Ukraine's Andriy Shevchenko glanced a header against the bar, while in reply Alexander Frei rattled the woodwork with a free-kick.
Competing in their first World Cup finals, Ukraine now meet Italy in the quarter-finals in Hamburg on Friday.
Shevchenko missed Ukraine's first penalty in the shoot-out, but as Marco Streller, Tranquillo Barnetta and Ricardo Cabanas failed with tame efforts, the Eastern European side progressed to the last eight.
Ukraine goalkeeper Olexandr Shovkovskiy saved the penalties of Streller and Cabanas, while Barnetta hit the bar with his spot-kick.
As they exited the competition Switzerland set two records.
They are the first team to go out of the World Cup without conceding a goal and they are the first side not to score a single penalty in a shoot-out.
Given Ukraine had lost their opening group game 4-0 and this is their first World Cup, Oleg Blokhin's side have regrouped impressively to reach this stage of the competition.
In a tightly contested match both teams found it difficult to create clear openings in the penalty area, with the game's best chances coming from set pieces as well as some powerful long-range shots.
Raphael Wicky tested Shovkovskiy early on with a rising shot, while a hammer effort from Shevchenko flashed past the post midway through the second half.
In the absence of the injured Philippe Senderos, the task of keeping a check on Shevchenko fell to Johan Djourou.
It did not take Shevchenko long to slip Djourou's leash.
A darting run allowed Shevchenko to sneak ahead of Djourou and the Ukraine striker then showed great bravery as he threw himself at the ball to reach Maxim Kalinichenko's free-kick only for his effort to rebound off the bar.
Switzerland striker Frei showed he could be as threatening when his superbly flighted free-kick left the Ukraine bar juddering.
The ball rebounded to Barnetta, whose effort was less impressive, as he ballooned his shot wide.
Just past the half-hour Switzerland withdrew Djourou, replacing the Arsenal teenager with Stephane Grichting.
Before the game went into extra-time, Ukraine twice went close from corners, as Andriy Gusin flicked a header narrowly, while Pascal Zuberbuhler sliced an attempted punch.
Kuhn's side failed to score a single penalty in the shoot-out
Ukraine also claimed a penalty after Grichting enveloped himself round Andriy Voronin after the Swiss defender came off second best in an aerial duel.
In extra-time Grichting redeemed himself when he bravely cut out substitute Milevskiy's dangerous cross.
At the other end of the pitch Gusin made a superb blocking tackle just as Streller prepared to shoot from a promising position.
Given the recent spate of yellow and red cards in the tournament this game was an altogether less explosive affair, with referee Benito Archundia not booking a player until just before the hour.
Not that the game was without its ugly moments, notably when Andriy Nesmachniy was left writhing in agony after an ugly challenge from Cabanas.
Vladislav Vashchuk was also unhappy with a Cabanas tackle as the Swiss midfielder raked his studs down the Ukraine defender's shin.
Switzerland: Zuberbuhler, Djourou (Grichting 33), Magnin, Muller, Philipp Degen, Vogel, Cabanas, Wicky, Barnetta, Yakin (Streller 64), Frei (Lustrinelli 116).
Subs Not Used: Benaglio, Margairaz, Behrami, Coltorti, David Degen, Dzemaili, Gygax, Senderos, Spycher.
Ukraine: Shovkovskiy, Vashchuk, Nesmachniy, Tymoschuk, Shelayev, Gusev, Gusin, Kalinichenko (Rotan 75), Voronin (Milevskiy 110), Vorobey (Rebrov 93), Shevchenko.
Subs Not Used: Pyatov, Yatsenko, Yezerskiy, Nazarenko, Byelik, Shust.
Ukraine win 3-0 on penalties.
Ref: Benito Archundia Tellez (Mexico).
Fifa man of the match: Olexandr Shovkovskiy
Ukraine have now reached the last eight on their World Cup debut. They have not conceded a goal in their last three World Cup contests and have now kept a clean sheet in eight of their nine matches in 2006.
Ukraine have become only the third country in World Cup history to reach the quarter-finals despite losing a group match by a margin of at least four goals. In 1954, West Germany claimed the title after having lost 8-3 to Hungary and Cameroon reached the quarter-finals in 1990 despite a 4-0 defeat against the Soviet Union.
Switzerland have become the first team ever to go out of the tournament without conceding a goal. They are also the first country to miss all their spot kicks in a World Cup penalty shoot-out. Ironically these were the first penalties the Swiss had to take, since they have not received a regular penalty in all of their 26 World Cup matches.
Trivia stats source: Infostrada Sports