Forest slipped to their first defeat under Joe Kinnear at Gillingham.
The visitors began well and went ahead with Nick Barmby's first Forest goal, sweeping home after Gareth Taylor had nodded on Mathieu Louis-Jean's cross.
But Gillingham responded and got back on level terms when Patrick Agyemang thundered in a shot from 25 yards.
The Gills dominated after the break and Danny Spiller's close-range drive won it, before Forest had defender Alan Rogers sent off for violent conduct.
Forest should have gone ahead before Barmby's goal, with Andy Reid shooting wide from Marlon King's cross and King himself forcing Lars Hirschfeld into a fine save.
But after the on-loan Leeds star had fired them in front, Gillingham began to get a grip on the game, with Paul Smith an influential figure.
And they got their reward with Agyemang's sensational strike that flew past Forest keeper Barry Roche, before Mamady Sidibe and Andy Hessenthaler both went close.
After the break both teams went for the three points, Kevin James heading over for the Gills while Taylor had a goal ruled out for offside for Forest.
After Nicky Southall's free-kick had clipped the bar Gillingham finally took the lead, a scramble in the visitors' box giving Spiller the chance to slam in the winner.
It got even worse for Forest in injury time as on-loan Leicester defender Rogers was given a straight red card for a foul on James.
Gillingham: Hirschfeld, Southall, Ashby, Leon Johnson, Rose, Hessenthaler, Smith, Spiller, James, Agyemang, Sidibe. Subs: Crofts, Jarvis, Wallis, Pouton, Beckwith.
Nottm Forest: Roche, Louis-Jean, Morgan, Walker, Rogers, Impey, Thompson, Barmby, Reid, Taylor, King. Subs: Jess, Formann, Chopra, Sonner, Doig.
Referee: P Crossley (Kent).