Phil Jagielka's injury-time scorcher gave Sheffield United their first Premiership win and heaped the pressure on Boro manager Gareth Southgate.
Captain Jagielka salutes the home crowd after victory
Aiyegbeni Yakubu and Jonathan Woodgate both missed golden chances for Boro before Rob Hulse put the Blades ahead on the counter-attack after 35 minutes.
Yakubu did slot home just after the break to draw the visitors level.
But it was all the home team from then on and Jagielka's late 30-yard strike gave United a well-deserved win.
Southgate had questioned his defence's passion in last week's home defeat to Blackburn.
They got plenty of chances to prove him wrong as Sheffield United laid siege to their goal in the second half.
Woodgate will have impressed the watching England manager Steve McClaren with his calm defending but it proved futile in the end as Jagielka struck.
Centre-half Woodgate was not as impressive at the other end, side-footing Stewart Downing's cross wide from six yards in the first half when completely unmarked.
That chance came minutes after Yakubu had hit the post when put clean through by Lee Cattermole.
The visitors were left to rue those misses as Sheffield United scored shortly afterwards on the counter-attack.
Some dogged defending from the home team led to Colin Kazim-Richards slipping Hulse through and the former Leeds striker found the net for his third Premiership goal of the season.
Southgate's half-time team-talk had the desired effect with Boro drawing level just three minutes after the restart.
It was another goal on the counter-attack with Yakubu this time keeping his composure when one-on-one against goalkeeper Paddy Kenny to find the corner of the net.
But after that, Boro offered almost nothing, their best chance coming when substitute Gaizka Mendieta threaded a pass through to Yakubu but Chris Armstrong's strong tackle saved United.
Southgate had promised a more attacking performance but Downing and the recalled Mark Viduka were virtually anonymous.
And egged on by a rowdy Bramall Lane crowd, United pressed urgently forward in search of a winner.
Leigh Bromby almost got through on the right while Keith Gillespie's fierce drive had to be batted away by Schwarzer.
Danny Webber dribbled through the Boro defence but dragged his shot just wide of Schwarzer's left-hand post.
It seemed the match was destined to end in a draw until captain Jagielka's injury-time winner gave the Blades their first Premiership win since beating Newcastle in April 1994 and lifted them off the foot of the table.
Sheffield United boss Neil Warnock:
"I thought we deserved it. We've now got the monkey off our back and the season now starts from here.
"We said it after training on Friday. We wanted to get a win to get our season up and running, and that's exactly what we have done.
"I cannot put into words just how big this win is - it's massive."
Middlesbrough boss Gareth Southgate:
"There's no question that we had chances to win this game.
"After going behind the response I had from the players in the second half is what I wanted. I thought it was a healthy performance.
"The things we have done in training this week are starting to show through and that's the way we will be going.
"We didn't get what we deserved. I don't like losing matches, and when you lose a game like that then it is very cruel."
Sheff Utd: Kenny, Bromby, Jagielka, Davis, Armstrong, Gillespie (Nade 89), Leigertwood, Tonge, Alan Quinn (Kabba 77), Kazim-Richards (Webber 77), Hulse.
Subs Not Used: Morgan, Kozluk.
Goals: Hulse 35, Jagielka 90.
Middlesbrough: Schwarzer, Davies, Woodgate, Pogatetz, Taylor, Morrison (Mendieta 71), Cattermole, Rochemback, Downing, Viduka, Yakubu.
Subs Not Used: Turnbull, Parnaby, Maccarone, Bates.
Booked: Woodgate, Cattermole, Rochemback.
Goals: Yakubu 49.
Ref: P Walton (Northamptonshire).