Peter MacDonald's late goal sent St Johnstone through to the the Scottish Cup fifth round after a gritty encounter at Tynecastle stadium.
Hearts had the best of a scrappy first half but could not find a goal.
The game improved considerably in the second period, and Samuel went close for Saints while Hearts' David Obua conspired to miss from three yards out.
But MacDonald struck from close range with four minutes remaining to send the visitors through to the next round.
With the game being rescheduled after postponement on Sunday because of broken pipes, both teams knew that victory would be rewarded with a home tie against either Falkirk or Partick Thistle.
But this did not seem to have inspired the fans, with a turnout of just 6,261 at Tynecastle.
The game started in an open manner with both sides pushing forward, and it was Hearts striker Marius Zaliukas who had the first effort on goal, but his long-range drive flew wide.
It was proving to be a fast, frenetic encounter, but perhaps both sides needed to take a bit more time with the final ball when attacking, as moves broke down in the final third.
Once Hearts settled, Arvydas Novikovas flashed the ball across the face of goal, but it was too far away from Kevin Kyle who was set to pounce inside the six-yard box.
The home side were creating more and Stephen Elliott fired the ball fiercely across the goalmouth after picking off a sloppy Chris Millar pass, but again Kyle could not connect with the cross under pressure from Michael Duberry.
Saints carved out a rare chance in the home box when Sam Parkin turned and shot, but the big striker put the ball wide of the mark.
And Samuel then saw his header from Millar's cross saved by Hearts goalkeeper Jamie MacDonald as Saints ended the opening period with a late flurry.
Right-back Dave Mackay took advantage of some slack Hearts defending to head the ball across the face of goal, but no Saints attacker had gambled and the home side escaped losing a goal, although the home defenders did not escape the wrath of manager Jim Jefferies.
But Grainger then had to spare his keeper's blushes as he headed the ball over his own bar after Peter Enckelman had flapped at a cross and missed it completely.
The game had improved immensely and was raging from end to end, but a poor first touch from Samuel left the Trinidadian facing a tight angle after being released in the Hearts box, and his shot went harmlessly wide.
The introduction of David Templeton galvanised Hearts as an attacking force and Enckelman had to look smart to snuff out a dangerous ball with Ismael Bouzid lurking in the box.
Fellow substitute David Obua then missed the best chance of the evening from three yards after Ruben Palazuelos had laid the ball across goal, but the Ugandan somehow scooped his shot over the bar.
It was a moment which seemed to sum up the match as both sides struggled to find a way past well-organised defences.
However, MacDonald pounced on Bouzid's failed attempt at a clearance to stab the ball into the net and seemingly claim Saints' place in the next round with his first goal of the season.
The drama had not ended, however, as Elliott lashed a close-range shot goalwards with seconds of injury time remaining only for Enckelman to pull off the save of the evening to deny the Irishman and eliminate Hearts from the cup.
Live text and stats
Tuesday, 11 January 2011
21 Ismael Bouzid
08 Black (Templeton 59)
09 Kyle (Obua 57)
18 Novikovas (Elliot 75)
03 Grainger yellow card
04 Morris yellow card
10 Craig (Maybury 89)
09 Parkin (MacDonald 78)
21 Samuel (May 66)
Ref: BrinesAtt: 6,261
Attempts on target
Attempts off target
90:00+3:08 The referee ends the match.
88:31 Substitution Alan Maybury joins the action as a substitute, replacing Liam Craig.
87:37 Stephen Elliott concedes a free kick for a foul on Murray Davidson. Daniel Grainger restarts play with the free kick.
85:38 The assist for the goal came from Steve May.
85:38 GOAL - Peter MacDonald:Hearts 0 - 1 St Johnstone Peter MacDonald grabs a goal from deep inside the penalty box to the bottom left corner of the goal. Hearts 0-1 St Johnstone.
84:15 Inswinging corner taken by Ryan Stevenson.
81:13 Foul by Ruben Palazuelos on Chris Millar, free kick awarded. Free kick taken by Daniel Grainger.
78:39 Outswinging corner taken from the left by-line by Daniel Grainger, clearance by Ismael Bouzid.
77:51 Substitution Peter MacDonald replaces Sam Parkin.
76:24 Corner taken by Daniel Grainger played to the near post.
74:58 Substitution Arvydas Novikovas goes off and Calum Elliot comes on.
72:19 The ball is crossed by David Templeton, save by Peter Enckelman.
68:11 Stephen Elliott takes a shot. Save by Peter Enckelman.
67:35 Centre by Ismael Bouzid, Peter Enckelman makes a save.
67:18 Stephen Elliott fouled by Michael Duberry, the ref awards a free kick. Direct free kick taken by Marius Zaliukas.
65:09 Sam Parkin is ruled offside. Jamie MacDonald takes the free kick.
65:09 Substitution Collin Samuel goes off and Steve May comes on.
63:44 The referee penalises David Obua for handball. Direct free kick taken by Daniel Grainger.
62:00 Close range shot by David Obua clears the bar.
60:34 Ryan Stevenson delivers the ball, save by Peter Enckelman.
59:56 David MacKay challenges Stephen Elliott unfairly and gives away a free kick. Direct free kick taken by Ryan Stevenson.
58:48 Sam Parkin gives away a free kick for an unfair challenge on Ian Black. Direct free kick taken by Ismael Bouzid.
58:48 Substitution David Templeton joins the action as a substitute, replacing Ian Black.
56:35 Substitution Kevin Kyle goes off and David Obua comes on.
55:52 Ryan Stevenson delivers the ball, save by Peter Enckelman.
53:28 Unfair challenge on Ryan Stevenson by Jody Morris results in a free kick. Ian Black takes the free kick. Arvydas Novikovas
takes the inswinging corner, Collin Samuel manages to make a clearance.
53:28 Booking Booking for Jody Morris.
52:07 Free kick awarded for an unfair challenge on Sam Parkin by Ismael Bouzid. Jody Morris takes the free kick.
48:59 Ian Black concedes a free kick for a foul on Collin Samuel. David MacKay takes the free kick.
47:31 The referee blows for offside. Indirect free kick taken by Jamie MacDonald.
47:09 David MacKay crosses the ball, clearance made by Ismael Bouzid.
45:53 Free kick awarded for an unfair challenge on Sam Parkin by Ismael Bouzid. Jody Morris takes the free kick.
45:01 The referee gets the second half started.
45:00+1:15 The ref blows to end the first period.
43:46 Ryan Stevenson crosses the ball.
43:19 The ball is crossed by Adrian Mrowiec, clearance by Steven Anderson.
36:57 Stephen Elliott fouled by Murray Davidson, the ref awards a free kick. Jamie MacDonald takes the direct free kick.
36:18 Stephen Elliott delivers the ball.
34:01 Unfair challenge on Ryan Stevenson by Murray Davidson results in a free kick. Ismael Bouzid takes the free kick.
Correction - 29:21 Foul by Daniel Grainger on Ian Black, free kick awarded. Shot on goal comes in from Arvydas Novikovas from the free kick.
29:21 Daniel Grainger concedes a free kick for a foul on Ian Black. Arvydas Novikovas restarts play with the free kick.
29:21 Booking Daniel Grainger receives a yellow card for unsporting behaviour.
27:02 Centre by Ruben Palazuelos, Sam Parkin makes a clearance.
25:46 Foul by Murray Davidson on Eggert Jonsson, free kick awarded. Free kick taken by Ismael Bouzid.
Correction - 24:10 Free kick awarded for a foul by Murray Davidson on Ian Black. Ismael Bouzid restarts play with the free kick. Kevin Kyle gives
away a free kick for an unfair challenge on Michael Duberry. Peter Enckelman takes the direct free kick. Liam Craig gives
away a free kick for an unfair challenge on Ian Black. Free kick taken by Jamie MacDonald.
24:10 Murray Davidson gives away a free kick for an unfair challenge on Ian Black. Ismael Bouzid restarts play with the free kick.
Kevin Kyle gives away a free kick for an unfair challenge on Michael Duberry. Free kick taken by Peter Enckelman. Liam Craig
concedes a free kick for a foul on Ian Black. Direct free kick taken by Jamie MacDonald.
22:46 Ruben Palazuelos produces a cross, clearance made by Michael Duberry. Corner taken by Arvydas Novikovas, clearance by Murray
22:04 The official flags Collin Samuel offside. Jamie MacDonald takes the free kick.
19:44 Unfair challenge on Kevin Kyle by Steven Anderson results in a free kick. The free kick is delivered right-footed by Ryan
Stevenson from right channel.
17:34 Steven Anderson fouled by Ryan Stevenson, the ref awards a free kick. Peter Enckelman restarts play with the free kick.
16:26 Arvydas Novikovas has an effort at goal from just inside the box that misses to the right of the goal.
12:05 Kevin Kyle concedes a free kick for a foul on Steven Anderson. Direct free kick taken by Peter Enckelman.
11:11 Ryan Stevenson produces a right-footed shot from just outside the box that goes wide left of the target.
10:45 Liam Craig takes a shot. Save by Jamie MacDonald.
8:46 Inswinging corner taken from the left by-line by Ryan Stevenson, clearance by Sam Parkin.
8:05 The referee blows for offside. Marius Zaliukas restarts play with the free kick.
Correction - 6:37 Corner from left by-line taken by Arvydas Novikovas.
6:37 Ryan Stevenson takes a outswinging corner.
0:00 The ref blows the whistle to start the match.
Live text and data provided by The Press Association.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.