"Gareth has given Boro magnificent service as a skipper and, in very difficult circumstances, as manager," added Gibson.
"I appointed Gareth in a situation that was greatly unfavourable to him. He is a good man and has all the qualities and integrity that we wanted in a manager.
"However, the time is right for change and that change has had to be made. Gareth will always be welcome at our football club. English football needs people of his stature and we feel certain that this experience will serve him well.
"Gareth deserves another opportunity once he has had the chance to rest and refresh himself."
Prior to Tuesday's win, Boro had lost three successive home games but their victory over Derby left them only one point behind leaders West Brom.
Richard Bevan, the chief executive of the League Managers' Association, said: "We are very disappointed for Gareth.
"Middlesbrough are only one point off an automatic promotion place and I believe it is a mistake.
"It's a demonstration of the pressures now at the top and the bottom of the table showing on administrators.
"The claim of concerns of a loss of form at home do not take into account excellent away wins at Sheffield Wednesday, Scunthorpe and Reading.
"Gareth is a manager learning his trade and Arsene Wenger has said he could be manager of England one day."
BBC Sport's Football League pundit Steve Claridge was also confused by the timing of Southgate's exit.
Claridge told BBC Sport: "There were times last year when he might have gone - he took an established Premier League side and got them relegated - but, having kept him over the summer, I just thought the brief was to see how things were at Christmas.
"You either sack him at the end of last season or give him the chance to take you back up - but Boro chairman Steve Gibson has done neither.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.