Substitute Craig Beattie fired a late goal to secure Scotland the points in Alex McLeish's first game in charge against an impressive Georgia side.
Beattie (right) celebrates his late winning goal at Hampden
Kris Boyd opened the scoring, heading home from a precise Gary Teale cross in the 11th minute.
Zurab Khizanishvili and Giorgi Demetradze went close for the visitors, before Shota Arveladze stooped to head a deserved equaliser after 39 minutes.
Scotland toiled until Beattie popped up to drill home an 88th-minute clincher.
With a trip to Italy looming in midweek, Scotland knew three points were needed if they were to sustain their qualification hopes from Group B.
McLeish opted to pair Boyd with Kenny Miller in a move to hopefully unsettle the visitors rearguard, and Celtic's Stephen McManus was awarded his second cap in the heart of the defence.
Both sides started energetically, and the onus was on the Scots to deliver, ahead of a tough task in Bari on Wednesday.
And McLeish - hoping for a victory in his debut game in charge - was treated to a perfect opener.
Teale had sufficient time and space on the right flank to curl in an inviting cross and, although Miller failed to connect, Boyd was on hand, unmarked, to head into the corner of the net from six yards.
It was the perfect start, and Teale should have added a second eight minutes later, but shot straight at Georgia goalkeeper Giorgi Lomaia when he found himself completely unchallenged in the danger zone.
The busy Miller then had a half-chance when he fired an angled shot which was well-saved by Lomaia.
The Georgians took heart from the scare to press forward, with Boyd's goal failing to unsettle their gameplan.
On the half-hour, Scotland goalkeeper Craig Gordon was called into action, turning away a Demetradze volley in a packed penalty area after former Rangers defender Khizanishvilli headed against the bar.
Kris Boyd turns away after heading Scotland in front
Demetradze and Levan Kobiashvili, the Schalke 04 midfielder, were becoming increasingly influential as the Scots struggled to retain possession.
Scotland, however, failed to heed the warnings, and it was no surprise when Klaus Toppmoller's team grabbed an equaliser in a move which the home defence should have countered.
Demetradze picked up possession on the left and David Weir allowed him time to curl in a cross which Graham Alexander should have headed clear.
The Preston North End defender missed his opportunity, and ex-Ranger Arveladze was lurking to convert a simple header beyond Gordon.
There was no question that the confident Georgians finished the half the stronger side, and their performace would have posed plenty of questions at the break for McLeish.
Scotland failed to take a grip after the interval, and McLeish elected to bring on Scott Brown for Teale in an attempt to freshen the midfield.
Levan Tskinishvili drove forward and threatened with a left-foot strike, with Scotland looking disjointed.
Boyd did carve out an opening, scuffing an effort against the bar after a link-up between Paul Hartley and Miller.
Seconds later, however, the Rangers striker inexplicably missed a glorious chance when he took a fresh-air swipe at Gary Naysmith's left-wing cross.
It was certainly a far from fluid performance and the home support appeared resigned to two points dropped, until Beattie - who had earlier replaced Boyd - came on to the scene.
With the Georgians failing to clear a speculative ball hovering in the penalty area, Beattie pounced and, as Lomaia advanced, the Celtic striker lashed home a low left-foot shot.
With France defeating Lithuania in Kaunas, Beattie's intervention proved even more important in Scotland's quest for qualification.
Scotland: Gordon, Graham Alexander, Weir, McManus, Naysmith, Teale (Brown 60), Ferguson, Hartley, McCulloch, Miller (Maloney 90), Boyd (Beattie 76). Subs Not Used: McGregor, Murty, Dailly, Severin.
Goals: Boyd 11, Beattie 88.
Georgia: Lomaia, Shashiashvili, Khizanishvili, Salukvadze, Eliava, Burduli (Siradze 57), Tskitishvili (Mujiri 90), Menteshashvili (Gogua 45), Kobiashvili, Demetradze, Arveladze. Subs Not Used: Revishvili, Kvirkvelia, Iashvili, Ashvetia.
Booked: Menteshashvili, Salukvadze, Demetradze.
Goal: Arveladze 39.
Ref: Nicolai Vollquartz (Denmark).