Steenson finally found his range in the 24th minute to open Exeter's account, expertly slotting over a penalty awarded after Northampton were penalised for collapsing the scrum.
The Chiefs stand-off added a further two penalties before the interval, but Reihana kept the scoreboard ticking over for the Saints with a third penalty of his own.
Exeter came out brightly after the restart and were almost rewarded for their efforts with a try, lock James Hanks stopped just short of the line.
Northampton continued to work hard and their second try came midway through the half as the pack drove over from a lineout, Tonga'uiha coming up with the ball.
But Exeter, as the league's newcomers, have proved themselves more than adept at competing in the Premiership, and were rewarded for their efforts with late tries from Budgen and Arscott to ensure they picked up a well-deserved losing bonus point.
Northampton director of rugby Jim Mallinder: "Our scrum went really well and we got a just reward from that in terms of penalties but it meant we could never really play from it.
"Our line-out didn't function as well as it had in the previous four games and we gave the ball away when we did have it.
"When you have that with struggling to defend and struggling to get the ball back it makes it a frustrating game, but there was no complacency there.
"I think Exeter came here with the intention of slowing things down a bit - if you noticed every scrum there was an injury, which is an interesting way to do it - but it worked for them."
Exeter Chiefs head coach Rob Baxter told BBC Radio Devon: "More than anything I'm pleased with our game management and how that improved from last week, we've kept telling the players that what we have to do is keep getting better.
"The bonus point is fantastic but the learning that we've made in the week was even better so I'm delighted for the players and the effort they've put in.
"We targeted 10 points from our first five games so we're probably now disappointed that we haven't got the 10, but that said nine points from this block of games is pretty good."
Replacements: Ansbro for Downey (62), Myler for Geraghty (57), Powell for Dickson (70), Dreyer for Tonga'uiha (65), Long for Sharman (65), Mujati for Murray (65), Sorenson for Lawes (57), Easter for Wilson (62).
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.