Hibs scored their third win over Rangers this season to knock the Ibrox side out of the Scottish Cup in style.
O'Connor opened the scoring for Hibs at Ibrox
Kris Boyd and Peter Lovenkrands went close with headers and Bob Malcolm had a long range shot saved by Simon Brown as Rangers dominated the first half.
But Garry O'Connor put Hibs ahead with a header after 50 minutes when he connected with an Ivan Sproule cross.
O'Connor then helped set up Sproule to drill in a low shot and new signing Chris Killen slipped in a third goal.
Hibs' top scorer Derek Riordan, who was the subject of a transfer bid from Rangers on Tuesday, was omitted from the starting line-up.
And Rangers welcomed back skipper Barry Ferguson who missed last weekend's visit to Inverness with a broken nose.
Sproule, who scored a memorable hat-trick on his last visit to Ibrox, stole a yard on the home defence in the opening minutes but shot wildly over the crossbar.
Hibs were dealt an early blow when midfielder Kevin Thomson left the game on a stretcher. Stephen Glass came on as his replacement after seven minutes.
Rangers were winning the battle in midfield and Thomas Buffel capitalised on a rash challenge from Steven Whittaker to burst down the left and centre for Boyd who sent a diving header wide of the target.
Boyd then sent a header over the top from inside the six yard box, with Hibs keeper Brown simply watching Buffel's corner.
Brown was then exposed as Lovenkrands glanced a header narrowly wide from a Chris Burke cross.
Sproule showed a few neat touches to evade a clutch of Rangers defenders but his sliced shot did not trouble Rangers goalie Ronald Wattereus.
Eight minutes before the interval, Malcolm thought he'd opened the scoring when he let fly from 22 yards but Brown was at full stretch to tip the ball on to the crossbar.
Boyd then passed up another headed chance when he stole in behind the Hibs defence only to skew his effort off target.
Five minutes after the restart Hibs went ahead after some good work from David Murphy and Sproule.
Sproule and O'Connor celebrate a fine win at Ibrox
The left-back broke forward and released his speedy team-mate who twisted past a couple of challenges and sent a cross to the back post where O'Connor rose to nod home.
Hibs were further ahead nine minutes later when O'Connor flicked a long ball into the path of Sproule and the Northern Ireland international buried a low shot in the corner of the net from 18 yards.
Rangers had the ball in the net soon after but Sotoris Kyrgiakos' effort was disallowed because Boyd had impeded Brown.
The home side were throwing everyone forward and pinning Hibs back around their own penalty area.
But Hibs, who last won the Scottish Cup in 1902, were sharp on the break and the visitors added a third goal after 77 minutes.
Riordan, on a substitute, threaded a a wonderful pass through to fellow replacement Killen, who tucked the ball in from a tight angle after Wattereus had blocked his first shot.
Only the fingertips of Wattereus denied O'Connor a second goal as the home side fell apart.
With Celtic already out, this will be the first time in 19 years that neither Old Firm side has reached the quarter-final stage.
Rangers: Waterreus, Ricksen, Andrews, Kyrgiakos, Murray, Burke, Ferguson, Malcolm (Prso 57), Buffel, Kris Boyd, Lovenkrands (Novo 83).Subs Not Used: Klos, Hutton, Smith.
Booked: Malcolm, Murray, Kris Boyd, Waterreus.
Hibernian: Simon Brown, Whittaker, Caldwell, Hogg, Murphy, Sproule, Thomson (Glass 7), Scott Brown (Killen 60), Stewart, Fletcher (Riordan 64), O'Connor. Subs Not Used: Malkowski, Shields.
Goals: O'Connor 50, Sproule 59, Killen 78. Booked: Thomson, Sproule, Riordan, Caldwell.
Ref: S Dougal