Ian Cox was sent off for a foul on Kevin Cooper but Jarvis turned things around for the Gills after 18 minutes.
Ki-Hyeon Seol might have equalised for Wolves two minutes later but Jason Brown got down well to save his header.
Gills' Darius Henderson missed a free header after the break but it did not matter as Wolves' Mark Kennedy wasted his chance to level in the 80th minute.
Wolves boss Dave Jones admitted his players just could not squeeze past a tough Gills' defence.
"When everything has gone so well in training and you have worked so hard it is frustrating for everyone involved," said Jones.
"They were always going to throw their bodies in front of the ball but we
didn't move it quickly enough.
"We allowed them to get back into a shape that was always going to be
difficult to break down. I'm very disappointed."
Gillingham: Brown, Nosworthy, Hope, Cox, Rose, Jarvis, Hessenthaler (Leon Johnson 65), Spiller, Crofts, Henderson (Tommy Johnson 61), Sidibe. Subs Not Used: Banks, Bodkin, Beckwith.
Sending Off: Cox (12).
Booked: Hessenthaler, Sidibe.
Goals: Jarvis 18.
Wolverhampton: Oakes, Lowe (Sturridge 45), Lescott, Bischoff (Craddock 60), Kennedy, Cooper, Cameron (Ince 90), Olofinjana, Newton, Cort, Seol. Subs Not Used: Murray, Clarke.
Booked: Sturridge, Ince.
Ref: T Kettle (Berkshire).