The Scots were punished for more slack defending in the final seconds of the first period as Van Persie raced on to powerfully head in a corner from Robben.
Kuyt screwed a low shot narrowly wide of the flailing McGregor's post after the interval.
McGregor, making his first competitive start for Scotland, then did well to get behind a clever, curling strike from the impressive Robben.
The Real Madrid star should have claimed a goal on 68 minutes but made a mess of his angled shot from close range after a flick on from Huntelaar, allowing Caldwell to scoop the ball awkwardly over his own crossbar.
Celtic defender Caldwell looked to have thrown Scotland an unlikely lifeline when he powered home a header from a Ross McCormack free-kick, but referee Laurent Duhamel ruled that Gary Teale had been impeding keeper Maarten Stekelenburg when there was little contact.
McCormack remains positive after defeat
Soon after, the French official, who is currently suspended by his home federation, further enraged the visiting fans when he awarded a penalty for Berra's trip on Huntelaar.
Liverpool forward Kuyt coolly rolled the ball in from the spot, with McGregor diving the wrong way.
The Scots, who have slipped to third in the qualifying section on goal difference, must now regroup and aim for maximum points against second-placed Iceland at Hampden on Wednesday.
Stekelenburg, Van Der Wiel, Mathijsen, Ooijer, Van Bronckhorst, Kuyt, Van Bommel, De Jong (Schaars 80), Robben, Huntelaar (Afellay 80), Van Persie (Sneijder 65). Subs Not Used: Timmer, Boulahrouz, Braafheid, Van der Vaart.
De Jong, Huntelaar.
Huntelaar 30, Van Persie 45, Kuyt 78 pen.
McGregor, Alexander (Hutton 73), Berra, Caldwell, Naysmith, Brown, Darren Fletcher, Ferguson, McCormack, Teale (Morrison 85), Miller (Steven Fletcher 71). Subs Not Used: Gordon, Barr, Iwelumo, Rae.
Laurent Duhamel (France).
BBC Sport Player Rater man of the match:
Netherlands' Robin van Persie (8.14 on 90 minutes).
Please note that you can still give the players marks out of 10 on BBC Sport's Player Rater after the match has finished.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.