Wales' returning strike duo of John Hartson and Craig Bellamy saw them to victory at the Skonto Stadium, Riga.
Bellamy was Wales' most potent attacking threat
Euro 2004 qualifiers Latvia had Wales under the cosh for much of the second half, but they defended doggedly.
Wales led when an 80th-minute Jason Koumas free-kick found Hartson, who shrugged off his marker to head in.
And with two minutes remaining, James Collins won the ball in midfield and fed fellow substitute Gareth Roberts who freed Bellamy to coolly slide home.
The game, the first meeting between the two sides, extended Wales' unbeaten run to five games ahead of their first World Cup qualifier in Azerbaijan (4 September).
Maris Verpakovski's speed up front troubled Wales in an end-to-end first half, Paul Jones having to be alert to the striker's runs.
The Wolves keeper was also called on to parry away a Juris Laizans free-kick from the left-hand side of the area, and to clear up after a dangerous Andrejs Rubins cross was met by Mihails Miholaps.
Wales' combative midfield ensured they were in the game, though they could be over-vigorous at times, Koumas, Robbie
Savage and captain Gary Speed all earning first-half bookings.
Bellamy was the man carrying the biggest threat, the Newcastle striker thriving up front alongside Hartson in a
new 4-4-2 formation.
His highlight was a dazzling 22nd-minute run from halfway, sizzling pace taking him to the Latvia box where a deflected shot nearly snuck past keeper Alex Kolinko.
Hartson had a great chance to put Wales ahead a minute before the break when Koumas' cross gave him a free header from 12 yards, but his effort slid tamely past the post.
Hartson relished the new 4-4-2 formation
After a quiet opening to the second half, a Hartson header created panic in the Latvia defence, Bellamy latching onto the loose ball but pulling a snap-shot across the face of the goal.
The visitors' defence then went to sleep, fluffing a free-kick clearance and forcing keeper Mark Crossley - who replaced Jones at half-time - to tip over a shot from Latvia replacement Igors Semjonovs.
Andrejs Prohorenkovs fired just over the bar after Wales were caught by a quick throw in, and Laizans went close as the pressure mounted.
Another Latvia free-kick deflected off the crossbar, sparking a furious scramble in the box that was somehow cleared by the ragged Wales defence.
It seemed only a matter of time before the home side scored, but Hartson's well directed header put the visitors ahead 10 minutes from the end.
And with Latvia chasing the game, the space opened up for Bellamy who finished off a sweeping move to seal the victory.
Latvia: Kolinko, Stepanovs, Lobanovs, Zemlinskis, Laizans,
Blagonadezdins (Korablovs 74), Isakovs,
Miholaps (Semjonovs 61), Verpakovskis, Rubins,
Prohorenkovs (Rimkus 66).
Subs Not Used: Piedels, Zakresevskis, Zirnis, Logins.
Booked: Blagonadezdins, Rimkus, Zemlinskis.
Wales: Jones (Crossley 45), Delaney, Melville (Collins 23),
Page, Thatcher, Koumas, Savage (Johnson 45), Speed,
Pembridge (Gareth Roberts 69), Hartson (Taylor 86), Bellamy.
Subs Not Used: Ward, Edwards, Fletcher, Robinson.
Booked: Koumas, Savage.
Goals: Hartson 80, Bellamy 89.
Referee: Ivan Ivanov (Bulgaria)