Morgan scored Gloucester's only try of the second half
Gloucester ended a run of nine away matches without a win stretching back to February with victory at Exeter.
Freddie Burns scored Gloucester's first try of the match after six minutes but Hoani Tui replied 12 minutes later.
Akapusi Qera went over three minutes before the interval to give the visitors a three-point half-time lead.
Olly Morgan's try seven minutes after the break gave the visitors breathing space, while Matt Jess's last-minute try secured Exeter the bonus point.
This was Gloucester's first victory away from Kingsholm since their
18-16 victory at Northampton
back on 26 February.
Burns fended off a couple of defenders to score the game's first try for the visitors on six minutes, before extended Exeter pressure enabled Tui to power over for the touchdown midway through the half.
Mike Tindall and Eliota Fuimaono-Sapolu combined to enable Qera to score Gloucester's second try to see them to half-time with a 14-11 lead.
After the break Morgan squeezed over in the left corner to extend the lead and put the game out Exeter's reach, but the home side's bonus point was secured when Chiefs replacement scrum half Kevin Barrett broke through to free Jess to go over.
POST-MATCH VIEWS
Exeter head coach Rob Baxter:
"The way that the last five of the game was going, when it looked increasingly like that game was slipping away, it was good to take the opportunity of that losing bonus point.
"Physically, again, we committed a lot to the game but the reality is that there were 23 or 24 turnovers.
"That is a real tough lesson for us. We paid the price for some sloppy stuff and came away with just the one point. We missed some regulation tackles and not really made them work for their tries."
Gloucester head coach Brian Redpath:
"Defensively we were really strong and if we had controlled some higher field position we would have scored more.
"Exeter are a strong scrum-and-drive, confrontational side so it was important that we did defend. I thought from one to 15, and also the subs that came on, we put in a big shift defensively.
"It was disappointing to give away a late try. We were going for the bonus-point try, we weren't winding down the clock, because we were going to play and win."
Bookmark with:
What are these?