Britain's number one said: "I got broken at the start of the second set and once I got broken he started to play a lot better.
"He managed to make me move a lot with his forehand but I still found a way to win. In five-set matches, the momentum always tends to shift.
"I was frustrated that I was playing well and let him back into the match. But I turned it around by playing more aggressive, playing the ball deep to his backhand and coming to the net a lot more."
The world number three has improved markedly on clay this year, having reached the semi-finals in Monte Carlo and the quarter-finals in Madrid, and few would have expected his mid-match wobble after the way he began this match.
The sparse crowd on Court Phillipe Chatrier witnessed a lively start from the Italian, who showed some deft touches at the net and baseline, but Murray's serve was a dominant force.
With Starace battling for every point to take his first two service games, the Scot had few problems to level at 2-2 and then at 30-30 in the fifth Murray scrambled to the net to make a sensational pass to grab the first break.
Starace will be kicking himself after throwing away his third-set lead
As Murray cranked up his booming forehand, Starace seemed to lose confidence and as the Parisian wind picked up Murray soon wrapped up the first set, dropping just one more game.
But then his wheels came crashing off. The signs were ominous in the second set's opening game when he had to fight back after facing his first break points.
Although the 22-year-old won that game, his focus appeared to slip and Starace took full advantage to take the following five games.
It was an astonishing turnaround as Murray looked increasingly desperate, with his recovery tactics including a number of racquet changes and he lost the set 6-2.
The 27-year-old Italian carried on in full-flow with his drop shot continuing as a dangerous weapon, and he raced to a 5-1 lead in the third set.
But an extraordinary 20 minutes then followed. Murray turned from no-hoper to world-beater again and after saving set points at 5-2 and 5-4, the seemingly impossible was achieved.
He won six successive games to take the set to leave his frowning opponent looking like a broken man.
Murray satisfied after tough win
The fourth set was a steadier affair with Starace regaining his form.
But as the match progressed, the accuracy of the Italian's drop shot wavered and Murray made no mistake on his third match point with a crashing cross-court forehand inside the line.
Murray was pleased to have coped with Starace's forehand, adding: "The most important thing is I found a way to win and I'm very happy to be in the next round."
It is the second year in a row he has made the third round in Paris at Roland Garros, having lost last year to Nicolas Almagro at the same stage.
Murray's next clash against Tipsarevic is likely to be on Friday, with the Serb, ranked 65 in the world, having a 2-2 record with the Scot.
The pair have never met on clay, although Tipsarevic beat him at Queen's Club in 2006.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.