Goals from Gregory Vignal and Nacho Novo gave Rangers a scrappy victory at Celtic Park that moves them three points clear of the champions.
Rangers had rarely threatened until Celtic goalkeeper Rab Douglas let defender Vignal's 25-yard drive slip through his grasp and into the net.
Opposite number Ronald Waterreus had been Rangers' hero, saving superbly from Craig Bellamy and John Hartson.
Striker Novo secured victory, lobbing Douglas with eight minutes remaining.
It ended Celtic's 11-game unbeaten run at home in Old Firm derbies and gave Rangers manager Alex McLeish his first victory at the home of his Glasgow rivals.
Celtic, who still have a game in hand, had won their last six meetings against the Ibrox outfit on their home pitch, including two already this season.
And they started confidently, with new signing Bellamy, on loan from Newcastle United, given his debut up front alongside Wales international colleague John Hartson, with Chris Sutton dropping into midfield.
It took Bellamy just four minutes to threaten, taking on Marvin Andrews before delivering a low drive that was held by Waterreus at the second attempt.
He had an even better chance after Hartson dispossessed Sotiris Kyrgiakos and sent his strike partner clear with only the goalkeeper to beat.
But Waterreus did well to beat away Bellamy's disappointing low drive from 16 yards.
Waterreus came to the rescue again when the ball fell to Hartson just inside the box, the Dutch goalkeeper making a brave block.
The same Rangers line-up had put four past Hibernian last week.
But, even with Barry Ferguson making his Old Firm return, McLeish's side found it hard going.
And the visitors did not really threaten Douglas's goal until 10 minutes after the break.
Dado Prso turned inside Neil Lennon only for the Celtic goalkeeper to beat away his powerful 18-yard drive.
Celtic pressed again and it took a great defensive header by Andrews to prevent Hartson pouncing from five yards out.
Hartson foxed Vignal at the edge of the Rangers box, but the striker's shot on the turn was again beaten away by Waterreus.
Rangers were beginning to dominate the midfield and Vignal, collecting a knock back from Fernando Ricksen, broke the deadlock, Douglas somehow letting the Frenchman's dipping drive slip through his grasp.
Novo added the second, pouncing on a moments' hesitation in the Celtic defence to latch on to a long ball from Ricksen and lob the ball over the advancing Douglas.
Ricksen appeared to be hit by a coin thrown from a spectator as full-time loomed.
Balde forced Waterreus into another save, but it was Rangers celebrating at the final whistle.
Celtic: Douglas, McNamara (McGeady 77), Balde, Varga, Laursen, Petrov, Lennon, Sutton, Thompson, Bellamy, Hartson.
Subs Not Used: Marshall, Henchoz, Juninho Paulista, Lambert, Maloney, Wallace.
Rangers: Waterreus, Hutton, Kyrgiakos, Andrews, Ball (Alex Rae 73), Buffel, Ferguson, Ricksen, Vignal, Prso, Novo.
Subs Not Used: McGregor, Namouchi, Burke, Malcolm, Thompson, Lovenkrands.
Booked: Andrews, Hutton.
Goals: Vignal 71, Novo 82.
Ref: M McCurry.