Norwich equalised with their first effort on goal after 33 minutes, Leon McKenzie pouncing from close range.
Rob Earnshaw put Norwich ahead on the stroke of half-time from Craig Fleming's cross, with keeper Robert Green then keeping the visitors at bay.
Sheffield United manager Neil Warnock:
"Our defending in the lead up to their goals left a lot to be desired.
"The lads are disappointed about that but overall they were fantastic for me.
"A performance was required and that's exactly what they gave me. If we keep playing like that for the final seven games I'll have absolutely no complaints."
Norwich: Green, Fleming, Shackell, Doherty, Drury, McVeigh (Safri 54), Hughes, Robinson, Huckerby (Charlton 89), Earnshaw, McKenzie (Johansson 69).
Subs Not Used: Gallacher, Etuhu.
Goals: McKenzie 33, Earnshaw 45.
Sheff Utd: Kenny, Kozluk, Short, Morgan, Unsworth, Tonge, Jagielka, Montgomery (Gillespie 80), Armstrong, Shipperley (Akinbiyi 80), Webber (Dyer 87).
Subs Not Used: Quinn, Lucketti.
Goals: Armstrong 17.
Ref: P Taylor (Hertfordshire).