St Johnstone recorded one of their finest results to stun Rangers at Ibrox and reach the CIS Cup semi-finals.
Milne (right) celebrates the first of his two goals at Ibrox
A Paul Sheerin corner broke to Steven Milne in the penalty area and he sent in a superb overhead kick from the edge of the six-yard box.
Milne then outpaced Brahim Hemdani to a long ball and drilled a shot beyond goalkeeper Allan McGregor.
With Rangers 15 points behind Celtic in the title race, the result puts further pressure on manager Paul Le Guen.
There was a minute's applause for former Rangers captain Bobby Shearer, who passed away earlier this week after a short illness.
However, the game ended with a chorus of angry jeers as Rangers' already miserable season took a turn for the worse.
St Johnstone were the better side and their first chance came from a Sheerin free-kick, which fell wide of the target before Jason Scotland tested McGregor.
Rangers found themselves under pressure when Scotland released Milne into the box but his run was halted by a timely interception from Steven Smith.
The visitors had an escape when Kris Boyd's shot from inside the six-yard box cannoned off the body of Allan McManus to safety.
MY SPORT DEBATE: Is it time for Le Guen to go?
Saints goalkeeper Bryn Halliwell then made a smart block from an angled Nacho Novo drive.
The underdogs were out of the traps quickly after the break and a long ball from Simon Mensing sent Scotland on a one-on-one with McGregor, who did well to block the shot.
But the Gers keeper could not prevent the First Division leaders taking the lead with 52 minutes gone.
A Sheerin corner broke to Milne in the crowded box and he conjured up a wonderful overhead kick to fire his side in front.
Boyd nodded wide from a fierce Lee Martin cross before the Saints added to their lead after 68 minutes.
Milne beat the sluggish Hemdani to a long ball before slotting under McGregor for his second goal.
The striker was denied a hat-trick when McGregor managed to paw his attempted lob away from a gaping goal.
A late effort from Boyd looked as though it had crept over the line before McManus hacked the ball away.
But no goal was given and Prso and Novo both earned themselves a booking for their protests to the assistant referee, summing up a dismal night for Rangers.
Rangers: McGregor, Hutton, Hemdani, Papac, Smith, Novo, Rae (Adam 70), Clement, Buffel (Martin 55), Sebo (Prso 54), Kris Boyd. Subs Not Used: Klos, Svensson.
Booked: Smith, Prso.
St Johnstone: Halliwell, Lawrie, McManus, Anderson, Stanic, Mensing, Lawson, Hardie (Sheridan 90), Sheerin, Scotland (MacDonald 84), Milne. Subs Not Used: Cuthbert, Jackson, Dyer.
Goals: Milne 52, 68.
Ref: S Dougal