"I knew that she'd be a tough player but I also knew that it was a winnable match. I thought I'd lost the match three times but I fought and fought.
"As long as I keep fighting I could have another surprise win."
The Scot's Wimbledon prospects looked gloomy after she was forced to pull out of qualifying for last week's Eastbourne warm-up because of back spasms, and got bleaker as Kerber raced to a 2-0 lead in the opening set.
But Baltacha broke straight back and settled into her rhythm, taking advantage of Kerber's hesistant serve to break her twice more to take a one-set advantage in just over half an hour.
Kerber threw her racquet to the ground in the opening game of the second but quickly regained her composure to break twice and level the contest.
Baltacha got back on track in a tense third set, carving out three break points in the opening game, but failing to take any of them.
It would be nice to win a match at Wimbledon and that's what I'm working towards
Both players struggled to build momentum, and each time Kerber broke Baltacha the Briton found an answer, breaking the German when she was serving for the match at 5-4.
Baltacha carved out a match point of her own in the 12th game, only to hit the ball wide, before making a second count thanks to a lucky bounce off the net.
And the Scot, who reached the third round of Wimbledon in 2002, sunk to her knees in celebration.
There were to be no celebrations for Bogdanovic as he failed to make good use of his Wimbledon wildcard for the seventh straight year.
The 24-year-old squandered a set point in the opener and then lost the tie-break to Botelli, the world number 46.
Bogdanovic has never won a singles match at Wimbledon
Bogdanovic claimed the second set to level the match and he and Bolleli went toe-to-toe in before the Italian broke twice in a tense third to seize the momentum.
The Briton, who is coached by Andy Murray's former mentor Brad Gilbert, was finally undone by his wayward play in the fourth set, seeing two set points sail by at 6-5 before losing the tie-break.
"It was a difficult match but I felt that it was close," said Bogdanovic, who has fallen to 242 in the world rankings but is ranked as Britain's number two.
"Every year that I play here I feel that I'm getting better and better and that's it's only a matter of time.
"It would be nice to win a match at Wimbledon and that's what I'm working towards - but it's disappointing today."
South put up firm resistance against Bondarenko but ultimately could not live with the Ukrainian's power and superior shot selection.
The 28th seed took the first set but South matched her in the second, taking it all the way to the tie-break which she roared through 7-1.
Bondarenko broke the Briton at 2-1 up in the third but South fought valiantly, saving four match points before finally conceding in just under two hours.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.