A sensational goal from Garry Wood two minutes from time sealed Berwick's win and earned them a Scottish Cup third-round home clash with Falkirk.
Berwick had been under a spell of intense pressure, but Wood lobbed goalkeeper Mark Peat from long range.
Earlier, strike partner Kevin Haynes gave Berwick a 35th-minute lead with fine shot into the top corner.
Arbroath peppered the Berwick goal with shots in the latter stages, but keeper Chris Flockhart was in fine form.
Berwick Rangers: Flockhart, Manson, Noble, Horn, Smith, McNicoll, Thomson, Fraser, Haynes (McLaughlin 74), Wood, David Greenhill. Subs Not Used: Shand, McCallum, Notman, Connon.
Booked: Horn, McNicoll, Thomson, Wood, Fraser. Goals: Haynes 35, Wood 88.
Arbroath: Peat, Rennie (Brazil 53), Watson, Raeside, Bishop, Smith, McMullen, Martin, Scott, Sellars, Stein. Subs Not Used: Reilly, Masson, Voight, Morrison.
Booked: Bishop, Scott, Watson.
Ref: J McKendrick