The Blues kept alive their hopes of a Heineken Cup play-off spot and a Celtic Cup place with a win at the Arms Park.
Blues (15) 22
Tries: Sowden-Taylor, Shanklin, Quinnell
Cons: Macleod (2)
Ulster (5) 8
Robin Sowden-Taylor got a first-minute try and Tom Shanklin followed him across the line, before Tommy Bowe pulled back a score for Ulster.
Shanklin limped off with a knee injury that could threaten his Lions hopes as the visitors staged a comeback.
But a 64th-minute Craig Quinnell try sealed the win to leave Ulster still sweating on a Celtic Cup spot.
The home side's celebrations were somewhat muted by the Shanklin injury and their failure to secure a fourth try for a bonus point.
Ulster were disorganised by the late injury withdrawals of David Humphreys and Tyrone Howe and knocked on from the opening kick off.
That allowed a swift attack from the home side, Ryan Powell and Shanklin quickly spreading the ball to the right and finding Sowden-Taylor who sprinted in past the disorganised defence.
Nick Macleod converted and added a penalty for a 10-point lead, but missed another chance after an excellent break from 19-year-old debut-making wing Chris Czekaj.
The youngster's quick thinking under a high kick then saw the ball spread to Shanklin on the left wing, the Wales three-quarter showing the pace to sprint past the shell-shocked visitors for the second try.
Paddy Wallace fired straight back for Ulster, his break setting up a forward drive that was held up over the Blues' line.
Czekaj had a promising debut for the Blues
The scrum five ball was sent to Tommy Bowe, the wing setting off on an arcing run and slipping Shanklin's tackle to open the scoring for his team.
Injuries disrupted both sides before the break, Rod Moore's removal prompting a front-row reshuffle for Ulster and Shanklin, who hopes to be named in the Lions party next Monday, trudging off.
Another Wallace break soon after half time resulted in a penalty for the fly-half to bring Ulster back to within a converted try.
The visitors' pressure increased, but a successful-looking Wallace penalty was ruled out by the officials and he then missed another long-range effort.
Having weathered the storm, the Blues eventually found their way into the Ulster 22 where a pop pass from the outstanding Martyn Williams sent Quinnell on a five-metre rumble to the line, his fellow forwards helping him to a crucial score.
"We were pleased to get the result, but a bonus point would have been nicer," said Blues coach Dai Young.
"We kept threatening to change gear but never managed it and they got back into it when they started hitting us up the middle.
"I hope the lost bonus point doesn't come back to bite us. We can only hope the other Welsh sides do us a favour."
Blues: Macleod, Czekaj, R. Williams, Shanklin, Morgan, N. Robinson, Powell, Yapp, G. Williams, Jenkins, Quinnell, Sidoli, Sowden-Taylor, M. Williams, Schubert.
Replacements: Davies for Shanklin (37), N. Thomas for Schubert (67). Not Used: Evans, R. Thomas, Budgett, Molitika, Dewdney.
Ulster: Cunningham, Bowe, Maggs, Steinmetz, S. Young, Wallace, Campbell, S. Best, Shields, Moore, Longwell, McCullough, Ward, Pfister, Wilson.
Replacements: Maxwell for S. Young (71), Larkin for Wallace (67), McCormick for Moore (33), Frost for Longwell (50). Not Used: Brady, Feather, Doak.
Referee: David Changleng (Scotland)