Ulster - who had not lost a game in 2010 - had started confidently with an excellent Darren Cave midfield break.
The home side eventually got out of their half with a high kick, and when Ulster fumbled the ball fell to the Scarlets' debut-making wing Joe Ajuwa.
The 26-year-old Nigerian, who has been starring for Llandovery in the Welsh Premiership, powered towards the visitors' line, only to be hauled down inches short by a desperate Ian Humphreys tackle.
Scarlets recycled, a fine pass from Knoyle - preferred at scrum-half to 2010 Wales Six Nations squad member Martin Roberts - switching the attack from right to left.
Joe Ajuwa makes a strong impression on his regional debut
The ball was moved quickly across the back line, giving wing Fenby an easy run to the corner for a try that was improved by Priestland.
Dan Tuohy crossed the Scarlets' line as the visitors made a quick riposte, but the flanker had overrun Nigel Brady in the build-up and the score was disallowed for a forward pass.
Ajuwa was penalised for a dangerous spear tackle on Brady in the same move, the visitors choosing to kick to touch but knocking on from the next attack to allow the home side to clear.
Another Ulster error proved costly as David Lyons broke off the back of a scrum 30m out.
The number eight delivered a back-handed pass to Knoyle, who slipped out of Tuohy's tackle and sprinted home.
Priestland missed the conversion, but made no mistake with a 27th-minute penalty as the home side stretched the lead to 15 unanswered points.
Ulster again killed the ball under pressure, receiving a severe talking to from referee Peter Allan as the home side chose to ignore the kick at goal and go for the try.
They repeatedly battered at the line, only to see that attack break down as Jonathan Davies dropped the ball in midfield.
The visitors hung on against more furious Scarlets play, then gave themselves hope with the last kick of the half, a fine 47m Humphreys penalty after opposite-number Priestland went over the top at a ruck.
Indiscipline costs Ulster as they receive two second-half yellow cards
The Belfast side were back within a score minutes into the second half, Isaac Boss splitting the defence with a fine show-and-go.
Prop Brendan Botha powered forward, and when the ball was quickly delivered to the three-quarters Wallace drove over from close range, Humphreys fluffing the conversion.
The Scarlets attacked from the restart and saw their opponents reduced to 14 men when flanker Pollock was adjudged to have killed the ball, Priestland landing the resultant penalty to make it 18-8.
The Scarlets brought Matthew Rees off the bench on 49 minutes, the British and Irish Lions hooker returning having missed all of Wales' Six Nations campaign to date with a groin injury.
Referee Allan further raised the ire of the visitors for denying them a fine attacking position with a dubious knock-on decision.
Knoyle came close to fashioning a try with a stunning break from deep in his own half, but Ulster's 14 men held out, the only points they conceded in the sin-bin period coming from the original penalty.
Scrum V post-match Scarlets analysis
The pressure continued after Pollock's return, though, and finally told as Priestland turned on the gas with an outside break, then an inside swerve to take him past full-back Jamie Smith for a try that the fly-half improved himself.
To compound their problems, Ulster were again reduced to 14 men as the game entered its final 10 minutes, Ryan Caldwell sin-binned for coming in at the side of a ruck.
But the Scarlets lacked the composure they had shown earlier in the game, failing to take advantage as they are still left seeking their first bonus-point league win of the season.
Scarlets coach Nigel Davies: "I'm very pleased - it's been frustrating that we've only been performing for half a game, but that was a full 80 minutes.
"It was a must-win game and for them to go out and play like that with that pressure on was magnificent. The youngsters are growing in confidence game by game.
"It builds momentum. We had a purple patch after Christmas followed by a few setbacks, but that was a great win against a very good Ulster side.
"We should definitely have got the bonus point but our driving maul let us down."
Scarlets: Newton, Ajuwa, King, J. Davies, Fenby, Priestland, Knoyle, I. Thomas, Owens, Manu, Reed, Day, Turnbull, Pugh, Lyons. Replacements: Roberts for Knoyle (75), John for I. Thomas (69), Rees for Owens (49), McCusker for Reed (66), Edwards for Pugh (70). Not Used: Evans, Maule.
Ulster: Smith, Nagusa, Cave, Wallace, Trimble, Humphreys, Boss, Court, Brady, Botha, O'Donoghue, Caldwell, Tuohy, Pollock, Henry, Faloon. Replacements: Danielli for Nagusa (50), O'Connor for Humphreys (62), Young for Court (62), Diack for O'Donoghue (65), Faloon for Pollock (65), Kyriacou for Faloon (75). Not Used: P. Marshall.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.