The Hatters took the lead after five minutes when striker Dean Morgan took possession and scored from 20 yards.
Sunderland levelled four minutes later when Daryl Murphy finished in the top corner from 12 yards for his third goal in as many games.
Connolly grabbed Sunderland's winner eight minutes after half-time with a clinical finish to seal the points.
Luton manager Mike Newell:
"I'm not having a go at the players for missing chances, I missed plenty myself, but they can come back to haunt you.
"We could have put the game beyond them but we didn't and we paid for that. You will never win anything if you don't take your chances when you are on top.
"We had plenty of the game. They didn't put us under any sustained pressure."
Sunderland: Fulop, Whitehead, Caldwell, Varga, Collins, Stephen Elliott, Miller, Kavanagh, Wallace, Connolly (Leadbitter 78), Murphy.
Subs Not Used: Carson, Nyatanga, Robbie Elliott, Yorke.
Goals: Murphy 9, Connolly 53.
Luton: Kiely, Foley, Heikkinen, Barnett, Emanuel, Edwards, Bell, Robinson, Brkovic, Vine (Feeney 76), Morgan (Boyd 76).
Subs Not Used: Beresford, Perrett, Langley.
Goals: Morgan 5.
Ref: A Penn (W Midlands).