Graham Bayne grabbed the points for Inverness Caledonian Thistle with a 90th minute goal in an incredible game.
Duncan rifled home the opening goal for on-form Inverness CT
Russell Duncan's low drive in 22 minutes put the visitors ahead, then John Rankin doubled the advantage from the penalty spot in 52 minutes.
Christophe Berra headed Hearts back into the match on the hour, but things looked bleak for the Jambos when keeper Eduardas Kurskis was sent off.
Andrius Velicka's penalty levelled the scores, but Bayne struck at the end.
The major talking point at kick-off was that Kurskis had retained his place in the Hearts goal, despite his basic error last week when he gifted Rangers the points at Ibrox.
It was a decision that was to backfire on Hearts.
With Kurskis having been yellow carded at the beginning of the second half for fouling Marius Niculae, he then became embroiled in a spat with Russell Duncan and was sent off for pushing the Inverness goalscorer with under five minutes remaining.
This Hearts fan might be wishing Santa brings a clean sheet for the Tynecastle men
Hearts assistant coach Stevie Frail, having used substitutes Calum Elliot, Michal Pospisil and Saulius Mikoliunas, instructed defender Lee Wallace to go in goals.
The home side started the match more brightly than the on-form Highlanders, who were searching for their fifth league win in a row.
Andrius Velicka forced a save from Inverness Caley keeper Michael Fraser with an angled shot in the sixth minute.
Fraser then parried a powerful shot from Kestutis Ivaskevicius in 10 minutes and moments later Audrius Ksanavicius thumped a dipping shot off the bar from 22 yards.
Inverness took the lead when Russell Duncan thundered a long-range shot past Kurskis to score only his second goal of the season.
The goal continues Hearts' unenviable record of having conceded in every home league match this season.
The visitors deserved their 1-0 half-time lead, with their crisp passing and team work characteristic of their play under returning manager Craig Brewster.
Michael Fraser saved well to his right from a 25-yard Velicka shot on 51 minutes.
Moments later Marius Niculae found himself one on one with Kurskis and was brought down. Rankin's penalty was directed into the corner of the net, leaving the Lithuanian keeper helpless.
Christophe Berra inspired a comeback for Hearts when he met Andy Driver's in-swinging corner to head home.
Hearts pushed forward and looked like they might draw level. However, with Kurskis being red carded the points looked to be heading north as the visitors looked to take advantage against 10 men.
Yet it was Hearts who scored next. Phil McGuire brought down Velicka in the box and the Hearts striker avenged the foul be scoring the equaliser from the penalty spot.
With seconds to go and the fans inside Tynecastle in a frenzy, Graham Bayne rammed home the winner with a right-footed shot from just inside the box.
Hearts: Kurskis, Neilson, Zaliukas, Berra, Wallace, Ivaskevicius (Mikoliunas 65), Jonsson, Palazuelos, Driver, Ksanavicius (Elliot 34), Velicka, Elliot (Pospisil 65). Subs Not Used: Basso, Goncalves, Cesnauskis, McGowan.
Sent Off: Kurskis (87).
Booked: Kurskis, Jonsson.
Goals: Berra 62, Velicka 90 pen.
Inverness CT: Fraser, Tokely, Munro, McGuire, Hastings, Cowie, Black (Wilson 81), Duncan, McBain (Bayne 70), Rankin, Niculae. Subs Not Used: Malkowski, McCaffrey, McAllister, Kerr, Gillespie.
Booked: Bayne, Duncan, McGuire.
Goals: Duncan 22, Rankin 53 pen, Bayne 90.
Referee: W Collum