Tommy Rowe earned Stockport their first win of the season after scoring four minutes into second half injury-time.
Rowe evaded several tackles before firing home from the edge of the area.
Stockport keeper Owain Fôn Williams was his side's hero in the first half with saves from shots by Richie Jones, Antony Sweeney and Andy Monkhouse.
County hit back after the break and Rowe was denied by Arran Lee-Barrett and also hit the bar before snatching his dramatic late winner.
Hartlepool boss Danny Wilson told BBC Radio Tees:
"I can't blame them at all. To have a goal go in in the 94th minute is a real body blow.
"Over the 90 minutes we played excellently at times. There was great enthusiasm, great endeavour, but we just couldn't get that goal."
"They put 11 men behind the ball and they made it very, very difficult but we still created the chances."
Hartlepool: Lee-Barrett, Sweeney, Nelson, Collins, Humphreys, Jones (Foley 68), Boland (Power 42), Liddle, Monkhouse (Barker 83), Brown, Porter. Subs Not Used: McCunnie, Robson.
Booked: Jones, Liddle, Humphreys.
Stockport: Fôn Williams, Mullins (McSweeney 46), Raynes, Owen, McNulty, Gleeson, Blizzard, Baker (Turnbull 70), Dicker, Rowe, Anthony Pilkington. Subs Not Used: Rigby, Thompson, McNeil.
Goals: Rowe 90.
Att: 3,945
Ref: Graham Salisbury (Lancashire).
~RS~q~RS~~RS~z~RS~12~RS~)