Barry Robson scored a penalty winner as Scotland's title race pendulum swung back Celtic's way after a dramatic win that extends their lead to five points.
Scott McDonald looked offside as he fired Celtic into an early lead.
Two Steven Davis corners were headed home by David Weir and Daniel Cousin for Rangers, but a McDonald drive looped in via a deflection.
And, when McDonald won a penalty, Robson fired home, after which Rangers' Steven Whittaker was shown a red card.
Rangers still have three games in hand despite back-to-back Old Firm derby defeats.
But the reigning champions could have increased their lead by another three points by the time their Glasgow rivals face Hibernian on Sunday should Gordon Strachan's side defeat Motherwell 24 hours earlier.
Cousin had been preferred to Jean-Claude Darcheville up front for Rangers.
It appeared that the Frenchman was being rested with Thursday's second leg of the Uefa Cup semi-final in mind.
But his replacement immediately looked to justify his selection, with goalkeeper Artur Boruc having to beat away the striker's drive on the turn from 12 yards following the first corner of the match.
Celtic responded to the early scare by taking the lead after four minutes.
With the influential Carlos Cuellar suspended, Christian Dailly had moved into the heart of the Rangers defence.
MY SPORT: DEBATE
And his partnership with fellow veteran David Weir was cut open when Jan Vennegoor of Hesselink rose to head the ball into the path of McDonald.
The little Australian was marginally offside as he gathered the ball before firing low past Neil Alexander from the edge of the penalty box.
But there was no whistle to cut short the celebrations by most of those packing Celtic Park.
The match was swinging from end to end as Rangers defied their recent reputation for negative tactics.
Celtic manager Gordon Strachan had made four changes from the side that defeated Aberdeen.
That effectively meant a return to the side that defeated Rangers 11 days earlier.
But, with Barry Ferguson fresh following his European suspension, Rangers were closing down their opponents far more effectively this time.
And the visitors drew level after 17 minutes when Weir rose highest in a packed penalty box to glance a Davis corner into the net from 10 yards.
Both sides were going for it now and Celtic were almost ahead again when Alexander scooped Shunsuke Nakamura's wicked curling free-kick off the goalline.
However, it was Rangers who edged ahead after 29 minutes.
Cousin this time rose unchallenged at the front post to head home off the underside of the crossbar.
Rangers fell two minutes short of retaining the lead at half time, McDonald's shot on the turn from 16 yards taking a deflection off Dailly and looping over Alexander's head.
Weir was forced off through injury five minutes after the break and, with six first-team players missing through injury, Walter Smith was forced to hand Amdy Faye his first appearance since October.
Nacho Novo sent a powerful effort just wide on the break, but the pendulum was swinging Celtic's way with Rangers defending more deeply.
And, when Broadfoot clumsily hauled McDonald down just inside the box after 70 minutes, Robson sent Alexander the wrong way from the spot.
Broadfoot was perhaps fortunate to escape a red card, but team-mate Whittaker was sent off in the dying seconds for a second yellow card as Celtic comfortably saw out the final minutes.
Celtic: Boruc, Hinkel, Caldwell, McManus (O'Dea 66), Naylor, Nakamura, Hartley (Scott Brown 83), Robson, McGeady, Vennegoor of Hesselink (Samaras 68), McDonald.
Subs Not Used: Mark Brown, Caddis, Donati, Wilson.
Booked: Robson, Hartley, Naylor, Vennegoor of Hesselink.
Goals: McDonald 4, 43, Robson 70 pen.
Rangers: Alexander, Broadfoot, Dailly, Weir (Faye 51), Papac, Whittaker, Hemdani (Darcheville 79), Ferguson, Davis, Novo, Cousin.
Subs Not Used: Graeme Smith, Boyd, Lennon, McMillan, Fleck.
Sent Off: Whittaker (90).
Booked: Broadfoot, Whittaker.
Goals: Weir 17, Cousin 29.
Att: 58,662
Ref: C Thomson
BBC Sport Player Rater man of the match: Celtic's Scott McDonald 8.02 (on 90 minutes).
• Please note that you can still give the players marks out of 10 on BBC Sport's Player Rater after the match has finished.
Player Rater~RS~q~RS~~RS~z~RS~49~RS~)