Kilmarnock somehow got the better of Inverness in a thrilling match that the home side dominated for long spells.
Steven Naismith's penalty proved the winner for Kilmarnock
Captain Darren Dods shot Inverness into an early lead only for Steven Naismith to level with a header and Colin Nish then drilled the visitors ahead.
Dods knocked in from close range and Craig Dargo converted a penalty to edge the home side in front at the interval.
But Nish picked out a second strike and Naismith claimed his double from the penalty spot.
The victory takes Kilmarnock up to fourth in the Premier League, while Inverness Caley Thistle's winless run is now up to eight games.
Charlie Christie's side started at a whirlwind pace and raced into an early lead.
Former Killie striker Dargo led the charge and when his fourth-minute strike took a deflection off Grant Murray, goalkeeper Graeme Smith made an outstanding save.
Dargo drove just wide, before the home side took the lead courtesy of skipper Dods, who was returning from suspension. Wilson flighted in a fantastic free-kick and Dods was on hand to lash a rising shot into the roof of Smith's net from close range.
Dennis Wyness saw his effort saved by the legs of Smith and the busy Killie keeper had to push away a free kick from John Rankin.
The Rugby Park side fashioned an undeserved equaliser when Hay's corner was was helped on by Gordon Greer and Naismith helped the ball over the line.
And the visitors found themselves ahead when Paul Di Giacomo set up Nish to wrong-foot Mark Brown with a neat strike. But the home side surged back with a Wyness header which struck the post, before a poor piece of goalkeeping saw parity restored.
Rankin's free-kick from almost 40 yards was fumbled by Smith and Dods was able to smash the ball into the net. On the stroke of half-time Dargo slammed in a penalty, after Nish was punished for blocking a Dods shot with his hand.
In the 50th minute Di Giacomo sent in a low cross which was expertly turned into the net by Nish from six yards.
Killie keeper Smith made a strong save from an Ian Black header and Rankin flashed a low drive into the side netting from 20 yards.
Inverness substitute Zander Sutherland then pushed Naismith in the area to concede a penalty and the Scotland Under-21 international picked himself up to drive the ball past Brown from 12 yards.
Nish should have completed his hat-trick with nine minutes remaining but blazed a shot over from only six yards. However, there were no late twists in a remarkable match and Kilmarnock held on to win.
Inverness CT: Brown, Tokely (Bayne 74), Dods, Munro, Hastings (McCaffrey 46), Wilson (Sutherland 66), Black, Rankin, McBain, Dargo, Wyness. Subs Not Used: Fraser, Morgan, Duncan, McAllister.
Booked: Wyness, Dargo. Goals: Dods 6, 39, Dargo 45 pen.
Kilmarnock: Smith, Greer, Frazer Wright, Grant Murray, Hay, Fowler, Invincibile, Johnston (Ford 85), Nish, Naismith, Di Giacomo. Subs Not Used: Harpur, Wales, Dodds, Leven, Hamill, Campbell.
Booked: Frazer Wright, Hay. Goals: Naismith 29, Nish 35, 50, Naismith 68 pen.
Ref: J Underhill