Andrew Caddick also bowled well to take the other three wickets, all of them coming in a single over, as he finished with 3-50.
For New Zealand only Daniel Vettori and Craig McMillan batted well, scoring 42 and 40, as the hosts struggled in Christchurch.
Vaughan is bowled for a duck
It's the first time Hoggard has taken five wickets in a Test innings, and on a difficult pitch England's lead of 144 already looks tough to beat.
Once again England lost an early wicket, this time Michael Vaughan for a duck, but Marcus Trescothick and Mark Butcher played well to help them recover.
Trescothick was out for 33, but Butcher on 22 not out and Nasser Hussain on six not out took England safely to the close.
Hear story
Darren Gough interview
Chat about sport
Try our cricket quiz
