Mark Schwarzer saved an injury-time penalty as Middlesbrough earned a place in next season's Uefa Cup.
Robbie Fowler missed the kick which would have seen City qualify at the expense of Boro.
Jimmy Floyd Hasselbaink's brilliant 30-yard free-kick put Boro ahead, but Kiki Musampa levelled after the break.
City boss Stuart Pearce brought on Nicky Weaver and pushed goalkeeper David James up front in a frantic finale, but Boro held out.
It was a dramatic end to a match in which City dominated possession without fashioning clear chances against a determined Boro back line.
The visitors seemed content to invite City to attack from the start.
Their first real chance came when Shaun Wright-Phillips burst through and lofted the ball over Mark Schwarzer, only to see it drift wide while Middlesbrough's sole chance in the early stages were limited to a long shot from Boudewijn Zenden.
Nevertheless, it was the visitors who broke the deadlock in spectacular fashion.
Hasselbaink's brilliant free-kick in the 23rd minute curled and swerved before cannoning in off the underside of the bar.
City responded well and continued to look threatening, particularly down the right-hand side and also through Wright-Phillips who saw another shot just clear the angle of post and bar.
But for all their huff and puff, there was precious little end product and City were pleased to see a long shot from Doriva flash wide.
City boss Stuart Pearce needed a stirring half-time team talk and his words had a remarkable effect.
The home side were level within 59 seconds of the restart, Musampa taking full advantage of some hesitant Boro defending to weave his way into the box and slot home.
The crowd played their part in firing up the City players as Boro found themselves on the back foot - although their defence was well-marshalled.
There were some fractious moments on the field in the passionate atmosphere and Ray Parlour was substituted after narrowly avoiding a second yellow card for a foul.
The atmosphere reached fever pitch late on as Pearce replaced Claudio Reyna with keeper Weaver and moved James into a centre-forward role.
And the game took a dramatic twist as referee Rob Styles penalised Franck Queudrue for handball - only for Fowler to miss the chance to fire City into Europe.
Man City: James, Mills (Croft 84), Onuoha, Distin, Thatcher, Shaun Wright-Phillips, Barton, Reyna (Weaver 88), Musampa, Fowler, Sibierski (Bradley Wright-Phillips 54).
Subs Not Used: Macken, Jordan.
Booked: Barton, Distin.
Goals: Musampa 46.
Middlesbrough: Schwarzer, Parnaby, Ehiogu, Southgate, Queudrue, Parlour (Morrison 69), Boateng, Doriva, Zenden, Nemeth (Downing 54), Hasselbaink, Morrison (Reiziger 84).
Subs Not Used: Cooper, Knight.
Booked: Queudrue, Zenden, Boateng, Parlour.
Goals: Hasselbaink 23.
Ref: R Styles (Hampshire).