But the following season, having signed Juve's two best players in Zlatan Ibrahimovic and Patrick Vieira, they cantered to the title.
They saw off Roma to make it three championships in a row the following season, but Mancini, widely regarded as one of the brightest young coaches in Europe, was sacked in the summer of 2008 and replaced by Jose Mourinho.
During an 18-month absence from the game, the 45-year-old Mancini has been linked with numerous managerial vacancies in a whole host of countries, but England has long been his first choice.
And, although his arrival at Eastlands in place of Mark Hughes came as a shock to many, City fans will be heartened to hear that he has been awaiting this opportunity for some time.
"He has been working hard over a year and half to learn English and has been living in London a few months to learn the language," said Mancini's close friend and former Sampdoria team-mate Gianluca Vialli.
"He has always been attracted by the style of football in the Premier League. I know he's particularly happy to be joining Manchester City.
Mancini enjoyed a glittering playing career with Sampdoria, Lazio and Italy
"He's been a top class player and he's quite experienced as a manager."
There is plenty of anecdotal evidence to prove that a glittering playing career does not necessarily translate into a similarly fruitful life in the dugout, but Mancini is not one such example.
A beautifully creative forward, he helped Sampdoria to a Serie A title, four Italian Cups, a Cup Winners' Cup and the 1992 European Cup final.
Another Scudetto, two more Italian Cups, a second Cup Winners' Cup, a European Super Cup and a Uefa Cup runners-up medal followed with Lazio.
Having scaled such heights with two sides from outside Italy's 'big four' - Juventus, Milan, Inter and Roma - Mancini is unlikely to be daunted by the task of achieving similar results at Eastlands.
Nor will a man who has managed in Italy, where a coach's shelf life is even shorter than in England, be fazed by the lack of patience that saw predecessor Mark Hughes sacked by City after only 18 months in charge.
Roberto Mancini won't be better than Mark Hughes - Alan Hansen
He is thought to be close to City chairman Khaldoon Al-Mubarak and, therefore, will be fully aware of owner Sheikh Mansour demands.
In short, City must win trophies and establish themselves as the biggest club in the world. Quickly. And Arsene Wenger is not sure he is up to the task.
"He is a manager who has done excellent work in Italy and he will have a Formula 1 (car) in his hands," said the Arsenal manager.
"How quickly will he drive? I don't know. Certainly he will have all the needed players he wants."
Mancini is no stranger to big budgets - Inter have spent heavily under Massimo Moratti - and he will be granted substantial funds to improve City's squad in January.
Mancini's impressive track record in the transfer market was underlined by the reluctance of Mourinho to make significant alterations to the squad he inherited.
City have already been linked with the likes of Fernando Torres, Franck Ribery, Sergio Aguero and Thierry Henry, but Mancini prioritises the team over individuals, and at Inter he reversed a previous policy of signing big names who consistently failed to produce results.
Mancini had a short taste of English football with Leicester in 2001
The former Italy international tends to line his teams up in a 4-3-1-2 formation but has experimented with pretty much every strategy apart from a three-pronged central defence.
A combative midfield trio forms the heart of his team, but whereas he could call upon Javier Zanetti, Esteban Cambiasso and Vieira at Inter, City's only real midfield destroyer is Nigel de Jong.
So Mancini is expected to recruit in that area - Barcelona's Yaya Toure is rumoured to be a target - and also reinforce a leaky defence.
But he will be delighted to learn that City are already well-blessed in the wing positions, which are pivotal to his tactical approach.
He likes to field at least one - often two - tall and robust strikers, meaning a department that consists of Emmanuel Adebayor and the injury-prone Roque Santa Cruz might well be bolstered.
The big question mark surrounding Mancini is his undistinguished managerial record in Europe.
City need not worry about that this term, but if they qualify for next season's Champions League and Mancini is still at the helm, they will be led into it by a man who lost his last job for failing to make it past the quarter-finals of that competition in three attempts.
Inter kept only one clean sheet at the San Siro in six knockout ties under Mancini, and two of the four matches they lost were on away goals.
In fairness, no coach has guided Inter to the European Cup since Helenio Herrera in 1965.
And none of that means Mancini will not flourish at the City of Manchester Stadium. Indeed, in Italy his appointment to one of the Premier League's top managerial posts has come as no surprise.
He joins England coach Fabio Capello, Chelsea manager Carlo Ancelotti and West Ham boss Gianfranco Zola as Italians working at the top level in England.
"Up until now the Italian coaches were quite reluctant to go outside the country," explained Italian Coaches' Association president Renzo Ulivieri.
"Just 10 years ago it would've been unthinkable and nobody would've considered so many of our tacticians working around the world.
"It means we are sought after because it's a sign of the quality and strong school of Italian football."
Mancini is well known as being a laid-back, personable individual with plenty of self-confidence and a good sense of humour.
He will be popular with his players, staff, fans and the media, but unless he brings City a first piece of major silver since 1976, his impressive credentials will count for nothing.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.