Jan Vennegoor of Hesselink struck in the final minute to snatch victory for Celtic before being sent off.
Vennegoor of Hesselink celebrates his winning goal
Derek Riordan was handed a simple tap-in when the Inverness defence got themselves into a tangle.
The home side hit back after the break when Graham Bayne headed home from a Grant Munro delivery.
But in injury time more bad defending allowed Vennegoor of Hesselink to shoot home before he was shown a second yellow card for rushing into the crowd.
The Dutch striker's celebration was deemed over zealous by referee Iain Brines and he was dismissed, having earlier been booked for disputing a throw-in.
And a scrappy game finished with both management teams exchanging heated words and a scuffle in the tunnel.
Inverness were without top scorer Craig Dargo, who could face another six weeks on the sidelines after dislocating a shoulder.
Within a minute of play, fellow forward Dennis Wyness was forced off with a similar injury.
Celtic had captain Neil Lennon and defender Stephen McManus back in the side and they were first to threaten when Vennegoor of Hesselink flashed an angled volley narrowly wide.
Rory McAllister replaced Wyness and the substitute fizzed in a shot from 20 yards which Celtic keeper Artur Boruc pushed away in less than convincing style.
Riordan was handed a rare start for Celtic and he claimed a penalty when his shot struck the outstretched arm of Ross Tokely.
Celtic were enjoying the bulk of possession but it was the home side who conspired to gift their opponents a goal after 37 minutes.
Steven Pressley moved out of defence to race down the right flank and deliver a cross which Richard Hastings intercepted at the near post but foolishly headed back into a crowded penalty box, while Roy McBain could only scoop an attempted clearance on to his own crossbar.
The ball fell kindly for Riordan to volley into an empty net.
Inverness responded well to the setback and a Graham Bayne header flew wide before John Rankin stung the palms of Boruc with a fierce shot from a narrow angle.
Celtic, who lost Pressley to a back injury in the first half, replaced Craig Beattie with Kenny Miller after the break.
But the Scotland striker was hurt immediately and limped off after 14 minutes.
Boruc made a good diving save to keep out a long range effort from Barry Wilson and Lee Naylor's sliding challenge prevented McAllister from reaching the loose ball.
The home side were level on 57 minutes when Munro was given time to pick out a cross and Bayne rose unchallenged to glance a header beyond Boruc.
A neat pass from Shunsuke Nakamura found Vennegoor of Hesselink with time to size up a shot from 18 yards but the striker's effort flew straight at keeper Mark Fraser.
However, the Dutchman was not to be denied and, with just a few seconds remaining, he lashed home a shot following more calamitous defending from the home side.
Celtic's joy was soured when the goal-hero was dismissed and manager Gordon Strachan will be concerned by the injuries picked up by Pressley, Beattie and Miller.
Inverness CT: Fraser, Tokely, Dods, Munro, Hastings, Wilson, McBain, Duncan (Black 74), Rankin, Bayne, Wyness (McAllister 2), McAllister (Paatelainen 84). Subs Not Used: Ridgers, McCaffrey, Morgan, Soane.
Goals: Bayne 57.
Celtic: Boruc, Telfer, Pressley (O'Dea 39), McManus, Naylor, Nakamura, Gravesen, Lennon, Riordan, Vennegoor of Hesselink, Beattie (Miller 45), Miller (Jarosik 60). Subs Not Used: Brown, McGeady, Sno, Maloney.
Sent Off: Vennegoor of Hesselink (90). Booked: Vennegoor of Hesselink, Jarosik. Goals: Riordan 37, Vennegoor of Hesselink 90.
Ref: I Brines