Mark Arber's last-gasp goal got Stevenage off the mark this season.
Defender Arber steered home Adam Miller's corner from the edge of the area deep into injury-time to deny last Histon a point.
Steve Morison had opened the scoring with a crisp finish after earlier hitting the bar with an angled shot.
But Chris Dillon restored parity in the 74th minute with a shot from the left hand side of the area before Arber notched the winner.
Stevenage: Julian, Fuller, Nutter, Arber, Henry, Miller, Cole (Dobson 35), Martin, Lewis, Bramble (Allen 42), Morison.
Subs Not Used: Russell, Oliver, Gaia.
Goals: Morison 43, Arber 90.
Histon: Osborn, Haniver, Okay, Mitchell-King, Ada, Barker, John Kennedy, Nightingale (Neil Kennedy 54), Knight-Percival, Wright (Dillon 67), Murray.
Subs Not Used: Naisbitt, Hipperson, Gwillim.
Booked: Mitchell-King, Okay, Haniver.
Goals: Dillon 74.
Ref: G Sutton (Lincolnshire).