The Sky Blues were ahead within five minutes of the start when striker Dele Adebola raced on to Kevin Kyle's flick before shooting past Kelvin Davis.
Southampton pulled level after a Gareth Bale corner, with Polish striker Marek Saganowski poking home from close range to score his first goal for the Saints.
Scot Kyle claimed Coventry's winner on 30 minutes, powering in a header.
Coventry manager Iain Dowie:
"There were a few weary legs out there tonight - there were lads who hadn't trained or played a lot. They have put in a huge shift against a top side.
"It's a big result, I thought it would be a tough game and so it turned out."
Southampton manager George Burley:
"I don't think we can just put it down to that. We were very poor first half.
"In the second, we just couldn't get the ball in the net. You have got to say the goals were individual errors, and that gave Coventry a lift."
Coventry: Marshall, Whing, Hawkins, Ward, Hall, McNamee (Osbourne 53), Hughes, Doyle, Tabb (McKenzie 73), Adebola, Kyle (Mifsud 81).
Subs Not Used: Steele, Virgo.
Booked: McNamee, Kyle.
Goals: Adebola 5, Kyle 30.
Southampton: Davis, Pele (Ostlund 46), Lundekvam, Baird, Bale, Surman, Wright, Viafara, Skacel (Belmadi 46), Saganowski, Jones (Rasiak 78).
Subs Not Used: Bialkowski, Wright-Phillips.
Booked: Ostlund.
Goals: Saganowski 14.
Att: 17,194
Ref: N Miller (Co Durham).