Chris Hall's 90th-minute goal booked Oldham an FA Cup second-round berth.
The Latics took the lead before the break when debutant Sean Gregan rifled in, only for Craig Mcilwain to level with a similar strike after the break.
Paul Warne poked home to restore Oldham's lead, and Neal Trotman's close-range finish made it 3-1.
Soon after Zema Abbey sidefooted home, before Brett Solkhon's looping header levelled the tie, only for Hall to have the last word with a late headed goal.
Kettering: Musselwhite, Nicell, McKie, Graham, Mcilwain, Solkhon, Hall, Boucaud (Howe 74), Abbey, Westcarr, Makofo (Caskey 69).
Subs Not Used: Okai, Marna, Bussey.
Booked: Boucaud, Mcilwain.
Goals: Mcilwain 52, Abbey 62, Solkhon 84.
Oldham: Pogliacomi, Lomax, Charlton (Tierney 7), Haining (Trotman 48), Gregan, Liddell, McDonald, Wellens, Taylor, Porter (Hall 81), Warne.
Subs Not Used: Wood, Terry Smith.
Goals: Gregan 43, Warne 56, Trotman 60, Hall 90.
Ref: N Swarbrick (Lancashire).