Substitute Wayne Andrews scored the only goal of the game just seconds after coming off the bench as Coventry nicked a late victory at Oakwell.
Andrews turned his marker to send a low shot into the far right corner with just six minutes remaining.
The home side dominated much of the second half in a game littered by missed chances.
The Tykes were inches away from a late equaliser but Andy Marshall pushed Daniel Nardiello's effort wide.
Barnsley boss Andy Ritchie:
"I've asked the lads if it's a fitness things but they say they felt good towards the end of the game," he said.
"Maybe it's a mental attitude but fitness goes hand in hand with that.
"We had chances in the second half which we should've put away but we haven't. Now we have to pull together and become stronger as a unit."
Coventry boss Micky Adams said he was amazed at the fans' initial reaction at the substitution which led to the game's only goal.
The supporters chanted "You don't know what you're doing" at Adams when Wayne Andrews replaced Leon McKenzie.
"We were playing like we wanted to but we lacked a bit of spark. I can understand a bit of the frustration," Adam said.
"They look at the league table and expect us to come here and turn Barnsley over, but it's not as simple as that."
Barnsley: Colgan, Hassell, Reid, Kay, Williams, Devaney (Healy 72), Togwell, Howard, McIndoe, Richards (Wright 82), Hayes (Nardiello 63).
Subs Not Used: Heckingbottom, Austin.
Coventry: Marshall, Duffy, Page, Ward, Clarke (Hall 27), Cameron (Tabb 60), Hughes, Doyle, McKenzie (Andrews 83), Kyle, Adebola.
Subs Not Used: Birchall, Heath. Booked: Doyle.
Goals: Andrews 84.
Ref: M Halsey (Lancashire).