Danny Webber opened the scoring after 93 seconds, dancing his way past Simon Walton and Paul Butler before drilling his first goal for the Blades.
Alan Quinn provided the second for midfielder Nick Montgomery to stab home from eight yards.
Rob Hulse hit the bar for Leeds before Andy Gray volleyed home a Phil Jagielka knock down then scored his 16th of the season at the second attempt.
Sheffield United manager Neil Warnock:
"You never expect anything in football, and I certainly didn't expect this. I thought the first half was as good as we have played for a number of years.
"It was a Premiership goal that gave the team a massive lift, and they just took off, reaching highs they've not reached this season.
"This was a great night for Sheffield United and we have to enjoy it."
Leeds: Sullivan, Walton (Gregan 39), Butler (Carlisle 81), Kilgallon, Pugh, Lennon, Derry, Johnson, King (Moore 53), Healy, Hulse.
Subs Not Used: Einarsson, Spring.
Booked: Lennon, Johnson, Gregan, Hulse.
Sheff Utd: Kenny, Kozluk, Morgan, Bromby, Harley, Quinn, Jagielka, Montgomery, Tonge (Geary 45), Gray, Webber (Kabba 84).
Subs Not Used: Liddell, Barnes, Thirlwell.
Goals: Webber 2, Montgomery 30, Gray 47, 74.
Ref: P Walton (Northamptonshire).