Colin Nish's tap-in with three minutes to go gave Kilmarnock an undeserved victory at Rugby Park.
Noel Hunt gave Dundee United the lead in the 50th minute when he swept home Barry Robson's deflected cross.
United were furious when Christian Kalvenes received a straight red card for a challenge on Steven Naismith in the 76th minute.
Moments later Kilmarnock substitute William Gibson equalised with a deflected shot before Nish's winner.
Naismith, Jamie Hamill and Ryan O'Leary, who celebrated call-ups for the Scotland Under-21 squad on Monday, were all in Kilmarnock's starting line-up at Rugby Park.
Gary Wales, who missed his side's goalless draw at Celtic on the opening day of the season, was also included in the side.
United, who opened their season with a 1-0 home victory over Aberdeen, welcomed Lee Wilkie back after suspension with Garry Kenneth dropping to the bench.
Scoring opportunities were at a premium in the early exchanges as both sides struggled for midfield supremacy.
Nish came closest to breaking the deadlock after 20 minutes, when his bullet header from Rhian Dodds was brilliantly cleared by Wilkie.
Allan Johnston was industrious at the heart of Kilmarnock's midfield and looked the most impressive player on the pitch in the opening quarter.
Naismith went close with a header from Nish's knock-down on the half-hour mark, but his effort drifted wide of Grzegorz Szamotulski's right-hand post.
Nish spurned a glorious chance a few minutes later when he too headed wide after a neat turn and cross from Johnston.
The visitors sparked into life following a superb run and shot by Craig Conway, whose effort was parried brilliantly by Alan Combe.
Noel Hunt tried in vain to knock in the rebound as the Kilmarnock defence cleared to safety.
The striker came close once again when his header from Barry Robson's inviting cross from the left sped past Combe's left-hand post.
The United keeper was fortunate not to concede the opening goal on the stroke of half-time when he flapped at Robson's in-swinging cross, but Hunt and David Robertson could not convert for the visitors.
Hunt gave United the lead four minutes into the second half when Robson's cross fell perfectly to Hunt and he fired home from inside the six-yard box.
Kilmarnock brought on Fernandez in an effort to restore parity, but the visitors looked the more threatening side.
Frazer Wright, earlier booked for a scything challenge on Prince Bauben, came close with a rasping drive which flew just over.
The game turned with 15 minutes remaining, when Kalvenes was harshly sent off for a challenge on Naismith when he appeared to make contact with the ball.
Moments later Gibson collected Johnston's cross and his deflected shot found a way past Szamotulski into the net.
And the home fans were in raptures when Nish steered home from inside the six-yard box with a couple of minutes remaining.
United manager Craig Levein was restrained by Kilmarnock assistant manager Billy Brown at full-time as the former tried to confront the referee.
Kilmarnock: Combe, Ford, Lilley, Wright, Hamill, O'Leary, Dodds (Fernandez 58), Johnston, Wales (Gibson 72), Nish, Naismith. Subs Not Used: Harpur, Locke, Murray, Di Giacomo, Bryson.
Booked: Wright.
Goals: Gibson 79, Nish 87.
Dundee Utd: Szamotulski, Dillon, Wilkie, Dods, Kalvenes, Conway (Duff 82), Hunt (Goodwillie 82), Robertson, Robson, Gomis, Bauben. Subs Not Used: McLean, Kerr, Kenneth, Proctor, Johnny Russell.
Sent Off: Kalvenes (77).
Booked: Dods.
Goals: Hunt 50.
Att: 5,557
Ref: B Winter
~RS~q~RS~~RS~z~RS~49~RS~)