Ugo Ehiogu scored his first goal for Rangers to put a small dent in Celtic's overwhelming lead at the top of the Premier League.
Ehiogu was the unlikely match-winner at Celtic Park
The veteran defender netted an impressive overhead kick early in the second period after Celtic had made all of the running in the opening half.
Aiden McGeady shot wide and Kenny Miller wasted two good opportunities.
Celtic pressed late on but Rangers held on comfortably for only their second win in seven years at Celtic Park.
Both sides made three changes after their midweek exertions in Europe.
Steven Pressley, Mark Wilson and Miller came in for Celtic, while Rangers goalkeeper Allan McGregor was back from suspension and Nacho Novo and Filip Sebo got rare starts.
Miller made a mess of a great early chance when he latched on to a Jan Vennegoor of Hesselink head-flick only to lob the ball high over the crossbar from the edge of the penalty area.
Evander Sno pulled an angled drive wide of the far post before Miller carelessly fired over from 12 yards after more good link play from Vennegoor of Hesselink.
Celtic continued to dominate and McGeady flashed a shot narrowly wide before he was unlucky not to win a penalty following a mistimed challenge from Ehiogu.
Rangers are now seven points clear in the race for second place
Shunsuke Nakamura worked a good one-two with Miller before his powerful strike was deflected over by the outstretched leg of Ehiogu.
Rangers were seldom seen as an attacking threat and their only sight of goal in the first half came when Sebo turned nicely on a Brahim Hemdani pass only to shoot tamely at Artur Boruc.
McGregor flapped at a Nakamura free kick soon after the restart but Sno could only head over from inside the six yard box.
Rangers then stunned the home support by taking the lead from an unlikely source on 49 minutes.
Celtic failed to clear a Charlie Adam corner and after a sequence of head tennis the ball fell to Ehiogu who executed a graceful overhead kick.
The diving Boruc appeared to have the shot covered but could only push the ball against the crossbar on its way into the net.
The Polish keeper continued to look shaky and a kick-out was nearly charged down by Sebo before he juggled awkwardly with an Adam free kick.
Rangers grew in confidence as Celtic's play became increasingly disjointed.
The home team did fashion a few late chances, with Vennegoor of Hesselink heading over and David Weir making a vital block to deny Derek Riordan.
Celtic: Boruc, Wilson, Pressley (O'Dea 78), McManus, Naylor, Nakamura, Lennon, Sno (Riordan 81), McGeady, Miller (Jarosik 74), Vennegoor of Hesselink. Subs Not Used: Brown, Beattie, Perrier Doumbe, Caldwell.
Booked: Lennon, Sno.
Rangers: McGregor, Hutton, Weir, Ehiogu, Murray, Hemdani, Thomson, Ferguson, Adam, Sebo (Prso 74), Novo (Rae 90). Subs Not Used: Klos, Boyd, Papac, McMillan, Furman.
Booked: Adam, Ferguson, Novo.
Goal: Ehiogu 50.
Ref: S Dougal