Mark de Vries controlled Jason Wilcox's cross and fired home to give the Foxes the lead and on the hour Stoke's Junior was sent off for an off-the-ball clash.
Stoke levelled when Marlon Broomes hooked in before Connolly took over, turning and firing home one goal before bursting clear and slotting coolly in.
Stoke boss Johan Boskamp:
"Joey Gudjonsson pushed into Junior and our player lashed out with his boot,
so the red card was correct.
"The players left on the pitch showed a lot of character to go ahead to chase a draw or the win.
"Even with 10 men, we went for it and it was a good game for the spectators. We played good football, created good chances and scored good goals."
Leicester: Douglas, Maybury, Johansson, McCarthy, Gilbert, Sylla (Hammond 71), Gudjonsson, Williams, Wilcox, Connolly, de Vries.
Subs Not Used: Henderson, Gerrbrand, Stearman, Wright.
Booked: McCarthy, de Vries.
Goals: de Vries 14, Connolly 66, 82, 90.
Stoke: Simonsen, Halls, Hoefkens, Duberry, Broomes, Chadwick, Junior, Brammer, Kolar, Russell (Dyer 40), Sidibe.
Subs Not Used: de Goey, Buxton, Henry, Taggart.
Sent Off: Junior (61).
Goals: Broomes 64, Halls 89.
Ref: P Taylor (Hertfordshire).