O'Neill set off in pursuit of referee Halsey at half-time
Mamady Sidibe claimed an injury-time winner to give Stoke a dramatic win over Aston Villa.
Sidibe was credited with the touch on Rory Delap's long throw to deny Villa, who twice battled back to level.
Delap was fouled by Martin Laursen to allow Liam Lawrence to fire Stoke ahead from the penalty spot, with John Carew scoring a well-crafted equaliser.
Ricardo Fuller's turn of pace and neat finish restored Stoke's lead, only for Laursen to level from close range.
Villa needed to don their tin hats as they came under siege from Stoke's direct style early on.
Interview: Stoke City boss Tony Pulis
The Premier League new boys took every opportunity to shell the ball into the Villa box, with Delap testing out keeper Brad Friedel's resolve with his artillery-distance throws.
Villa tried to take the sting out of Stoke by keeping possession but the Potters were relentless in hounding them down.
It took Villa 22 minutes to win their first corner but Stoke keeper Thomas Sorensen got a firm fist to Ashley Young's kick and his right hook was in action again as Young curled in a free-kick after debutant Abdoulaye Faye fouled Villa striker John Carew.
Referee Mark Halsey had to make two big decisions on the half hour, which ended with Stoke taking the lead.
He first waved away Villa appeals when Seyi Olofinjana appeared to bundle Agbonlahor to the ground, but seconds later was pointing to the spot at the other end, after Delap was clipped by Laursen as he turned tightly.
Friedel guessed correctly but could not prevent Lawrence scoring the first Premier League goal at the Britannia Stadium.
The Villa keeper then got lucky when he advanced outside his area and his attempt to chest the ball down fell to Fuller, only for Laursen to cannily fence the Jamaican into a tight shooting angle which saw him fire into the side-netting.
Fuller almost doubled Stoke's lead on the stroke of half-time when he got the better of Curtis Davies to meet another Delap howitzer throw and nodded just wide.
Interview: Aston Villa boss O'Neill
Villa boss Martin O'Neill showed little respect as he angrily pursued referee Halsey down the tunnel at half-time and his team certainly came out fired-up for the start of the second period.
But perhaps in answer to boss Tony Pulis's criticism of their defending at set-pieces in their opening day defeat at Bolton, Stoke handled any dead balls that came their way with relative comfort.
With the aerial route denied to them Villa's ground force picked their way through the Stoke defence to score a delightful equaliser.
Carew played the ball into Ashley Young who cutely back-heeled a return-pass into the Norwegian's path and he fired across Sorensen to bring Villa level.
Lawrence celebrates the first Premier League goal at the Britannia Stadium
Suddenly, Stoke were on the back foot and Gareth Barry was inches away from making contact with Ashley Young's cross.
But Fuller put Stoke ahead with a turn of pace fellow Jamaican Usain Bolt would have admired, as he moved on to Lawrence's pass, neatly flicked the ball round Laursen and outpaced the Villa defender to score from a tight angle.
Stoke's lead lasted all of two minutes as their inability to defend set pieces returned to haunt them.
Ashley Young's free-kick was poor but it squirted off a defender and fell kindly for Laursen who buried from eight yards.
Stoke were still keen on a winner and Delap launched another throw which saw Leon Cort land a header on the roof of the Villa net.
The Premier League new boys were not to be denied and Delap wound up another throw. With a clutch of players rising at the near post, Sidibe got an unwitting touch but was happy to claim the goal after the ball looped into the net.
Stoke boss Tony Pulis:
"We'll enjoy this but we know it will be difficult and we need the supporters to stick with us.
"We have to make sure that defeats don't get us down and that we don't labour over them.
"I don't want to single any players out as everyone played a part, and that's what will get us through this season."
Aston Villa boss Martin O'Neill on Rory Delap's throw-ins:
"They're very dangerous but we thought we had resolved the problems of coping with them.
"We had one last one to deal with and whether it was a bit of luck or what, the ball ended up in our net.
"One thing you shouldn't do in the last minute of a game, is if you can't win it, then at least don't lose it."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.