The Tigers came out a renewed force in the second period, and two early penalties from Staunton reduced the deficit.
But Exeter dug in as Foster ended a superb break to scramble over in the corner, Steenson converting the try to put the visitors 27-16 up.
But as the game reached its final quarter, Leicester stepped up the pressure on the Premiership newcomers even further and after Hamilton and Hipkiss went over in quick succession, the Tigers found themselves back in the lead.
A late penalty try, awarded after Exeter's scrum buckled, gave the reigning champions a bonus point, as they wrapped up their first win of the season.
Leicester director of rugby Richard Cockerill told BBC Radio Leicester: "Exeter will cause a lot of teams a lot of problems. They caused us a huge amount of problems today.
"You can't just take it that we played poorly. They deserve a huge amount of credit for what they did.
"Today we had the shock of our lives and it's been a reality check for everybody."
Exeter Chiefs forwards coach Robin Cowling told BBC Radio Devon: "It was a tremendous performance and obviously just at the moment we're very disappointed, one to have lost the game, and two not to come out with a bonus point.
"We've got to pick ourselves up from this - the season isn't over after two games.
"We now know we're good enough at this level, and we were on the edge of causing a real upset."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.