Yakubu notched his first Middlesbrough goal and Massimo Maccarone also scored as Arsenal lost for the first time in six league visits to the Riverside.
Nigerian striker Yakubu put his side ahead on 40 minutes, turning smartly to curl a shot into the bottom corner.
Boro grew in confidence after their opener and Maccarone added a second with a clinical finish on 59 minutes.
Jose Antonio Reyes, who hit the post in the first half, scored a late goal for Arsenal but it was a mere consolation.
Arsenal looked uncomfortable for much of the game, sorely missing their talismanic skipper Thierry Henry.
With the French striker out, the pressure was on Reyes to spearhead the attack.
It was a challenge the Spaniard was clearly relishing as he set about performing his own version of a one-man show.
In the opening four minutes he could have scored twice.
But once Boro had gained some composure they began to stroke the ball around confidently.
Yakubu was showing that lone strikers can still prove a handful - as Kolo Toure would testify as he struggled to contain him.
And after a few physical tussles between the pair, Yakubu found a way past his marker to open the scoring.
Yakubu was lucky with the bounce of the ball as he swivelled, but there was nothing fortunate about his curling right-foot shot into the bottom corner.
After the interval, the main protagonists Reyes and Yakubu quickly resumed their roles.
But while Reyes jinked and probed, Yakubu's direct approach was having a greater impact.
And he was only denied a second goal by a superb save by Lehmann, who somehow stretched out his left hand to turn the ball around the post.
Boro, who had been defensively dreadful in recent matches, were throwing themselves into tackles - and having a lot of success.
And it was just after they collectively stopped Alexsander Hleb in the penalty area, that they doubled their advantage.
With Arsenal pleading for a penalty, Boro launched a quick attack and the ball worked its way through to Maccarone via a tackle by Toure on Yakubu.
Maccarone made the chance look easy and Arsenal's hoodoo over Boro was broken, despite Reyes slotting home late on.
Middlesbrough boss Steve McClaren:
"In the last eight games they've beaten us but there's always an opportunity to turn that around and I really fancied we would today.
"The first 10 minutes were hectic for us and we were hesitant, while Arsenal dominated - and hit the post.
"But we survived that, got the break and got better and better. We defended well and we could have scored more."
Arsenal manager Arsene Wenger:
"We never should have lost. We should have shown more determination.
"I feel that we had very good patches in the game - we could have won this game in the first 20 minutes with three clear chances but we did not take them.
"We missed our chances and we gave them presents for their two goals. We were good enough to win this game."
Middlesbrough: Schwarzer, Xavier, Ehiogu, Southgate, Queudrue, Maccarone (Doriva 79), Parlour, Boateng, Rochemback, Johnson (Pogatetz 65), Yakubu (Graham 90).
Subs Not Used: Jones, Kennedy.
Booked: Boateng, Queudrue, Rochemback.
Goals: Yakubu 40, Maccarone 59.
Arsenal: Lehmann, Lauren, Toure, Cygan, Cole, Pires (Van Persie 72), Flamini (Fabregas 60), Silva, Hleb, Bergkamp, Reyes.
Subs Not Used: Senderos, Clichy, Almunia.
Booked: Cole, Flamini, Lauren, Fabregas.
Goals: Reyes 90.
Att: 28,075
Ref: M Riley (W Yorkshire)