Yakubu's winner ended Portsmouth's run of six League games without a victory.
Malcolm Christie, returning after four months out with a broken leg, put Boro ahead after Kostas Chalkias parried Boudewijn Zenden's shot into his path.
But another goalkeeper error, this time from Mark Schwarzer, allowed Portsmouth to equalise, Matt Taylor rifling home after his first shot squirmed free.
Chalkias then denied Christie after the break before Yakubu swept home Ricardo Fuller's cross for the winner.
Yakubu had looked dangerous throughout and deserved his goal for a hard-working performance.
He had the game's first meaningful effort on goal after 18 minutes, when he unleashed a low shot from the edge of the area that Schwarzer smothered.
And the Nigerian striker should have scored when he latched on to a Chalkias clearance only to see his shot trundle wide of goal.
At the other end, Christie had made a quiet start alongside Jimmy Floyd Hasselbaink before bursting into life on 35 minutes.
Zenden's low shot bounced off Chalkias, allowing Christie to smash home the loose ball for his first goal since September 2003.
Middlesbrough's lead lasted only five minutes.
Taylor's powerful drive escaped Schwarzer's clutches and the full-back fired his second shot into the roof of the net.
Both sides made a lively start to the second half, with Patrik Berger twice finding the side-netting and Chalkias blocking Christie's shot.
But the home side grabbed the winner on 58 minutes, Yakubu prodding Fuller's left0wing cross into the net.
Boro's response was to pour forward but the home defence stood firm to give manager Velamir Zajec only his second Premiership win.
Portsmouth coach Joe Jordan:
"We've played well over the last few weeks without collecting the points.
"Here we deserved the win and we got it. There have been too many games where that has not happened and we did not want another hard luck story.
"We showed there is a spirit here too. That is important because if you go behind you have to rally together."
Middlesbrough manager Steve McClaren:
"I'm so disappointed we've lost this game and we should not have done. We played well enough to get something out it.
"We were poor after our goal and made fundamental mistakes. Individual errors are costing us goals at the moment.
"We're not getting the rub of the green either but we have to soldier on because there is still a lot to play for."
Portsmouth: Chalkias, Primus, Stefanovic, De Zeeuw, Taylor, O'Neil, Cisse, Hughes, Berger (Rodic 81), Fuller (Skopelitis 73), Yakubu.
Subs Not Used: Curtis, Ashdown, Mezague.
Goals: Taylor 40, Yakubu 58.
Middlesbrough: Schwarzer, Reiziger, Southgate, Cooper (McMahon 29), Queudrue, Parlour, Morrison, Zenden, Downing, Christie (Job 66), Hasselbaink, McMahon (Graham 80).
Subs Not Used: Nash, Doriva.
Booked: Parlour, Zenden.
Goals: Christie 35.
Ref: P Crossley (Kent).