As well as having Marlon Broomes sent off, Stoke also had boss Johan Boskamp ordered from the dug out.
Walcott steered home from Nigel Quashie's pass in the 16th minute, having already had a goal disallowed.
In the 40th minute, Broomes saw red for lashing out at Walcott before Belmadi scored at the far post late on.
Stoke manager Johan Boskamp:
"Marlon Broomes' sending off was not a sending off, I'm 100% sure of that. The referee didn't send him off, the linesman did.
"The whole team feel very angry. It changed the whole match and I only asked the referee politely why he made the decision he did.
"He sent me to the stands because I left the technical area."
Southampton: Niemi, Delap, Lundekvam, Svensson, Higginbotham, Belmadi, Quashie, Wise, McCann, Walcott, Fuller (Jones 77).
Subs Not Used: Smith, Kosowski, Powell, Folly.
Goals: Walcott 16, Belmadi 90.
Stoke: Simonsen, Halls, Broomes, Hoefkens, Wilkinson, Russell, Mbuyi, Brammer, Sigurdsson (Sweeney 86), Gallagher (Paterson 83), Sidibe.
Subs Not Used: de Goey, Kolar, Henry.
Sent Off: Broomes (40).
Ref: R Oliver (W Midlands).