Hearts caretaker head coach Stevie Frail made four changes after last week's draw with Falkirk.
Anthony Basso took the place of virus-hit Steve Banks in goal, while Jose Goncalves, Christian Nade and Eggert Jonsson all returned to the starting line-up.
Marius Zaliukas met Michael Stewart's early out-swinging corner, but the Hearts defender's header drifted wide.
Ghana international Kingston then had United goalkeeper Zaluska worried with a cross-shot that landed on the roof of the net.
United striker Hunt was guilty of missing a sitter in the 19th minute when he took an air kick at the ball from eight yards from Willo Flood's cross.
But it was Kingston who broke the deadlock with a composed finish in the 27th minute.
Eggert Jonsson slipped the ball through to Kingston, who ran clear of the United defence before tucking the ball past Zaluska.
While the Hearts supporters came to watch the action on the pitch, they were temporarily distracted by a shouting match between caretaker boss Stephen Frail, from the dugout, and striker Christian Nade.
Frail appeared unhappy the player was not heeding team instructions.
Zaluska kept United in the game with a good save from Mikoliunas' header five minutes before the break.
Nade forced Zaluska to parry the ball clear after turning his marker and firing in a shot from 15 yards.
Palazuelos fired the ball wide from the edge of the box on the rebound.
Christophe Berra's last-ditch interception prevented Mark De Vries latching on to a high ball over the Hearts defence.
And, from the corner, Basso did well to hold Morgaro Gomis' shot from the edge of the box.
Mikoliunas missed a superb chance after Zaluska could only palm clear Kingston's low cross and the Lithuanian fired over from eight yards with the goal gaping.
Kenneth was sent off for a late tackle that led to a second yellow card as the match entered stoppage time and Swanson was shown a straight red card for a bad challenge.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.