Bury scored twice late on to inflict more misery on 10-man Sunderland.
Quinn is set to step down as manager soon
The visitors played most of the game with only 10 men after Arnau Riera was sent off with less than three minutes gone on his first start for the club.
Eight minutes from time Bury defender John Fitzgerald rose to thump a header past keeper Ben Alnwick from a corner.
Fitzgerald, who had already been booked, was then sent off after he was yellow carded for his celebrations, but an Andy Bishop goal sealed Bury's win.
Sunderland, who have lost their opening four Championship games, got off to the worst possible start against the Football League's bottom club when Riera was dismissed.
The Spaniard was shown the red card after catching Richie Baker with his arm.
Sunderland held on until the last 10 minutes but Alnwick was unable to keep out Fitzgerald's powerful header following Brian Barry-Murphy's corner.
Fitzgerald then jumped straight into the crowd and was sent off for his second yellow card.
But with four minutes left Bishop sprung the offside trap to clip the ball past Alnwick to compound Sunderland's miserable start to the season.
The game could be Niall Quinn's last in charge as boss as he revelaed after the game that he was on the verge of securing a new manager.
Bury: Fettis, Scott, Fitzgerald, Woodthorpe, Brass (Parrish 81), Baker, Adams, Buchanan, Mattis (Barry-Murphy 82), Pittman (Youngs 57), Bishop.
Subs Not Used: Collinge, Flitcroft.
Sent Off: Fitzgerald (83).
Booked: Fitzgerald, Brass.
Goals: Fitzgerald 82, Bishop 88.
Sunderland: Alnwick, Cunningham, Neill Collins, Robbie Elliott (Stead 86), Wright (Lawrence 38), Whitehead, Leadbitter, Riera, Delap (Danny Collins 70), Stephen Elliott, Murphy.
Subs Not Used: Ward, Smith.
Sent Off: Riera (4).
Ref: M Jones (W Yorkshire).