"I get on with him fine thank you very much, I think he's been doing an exceptionally good job in what are very very challenging conditions for our country," he said.
"Of course we've had our ups and downs but we've also known each other for over 20 years and originally we worked very well together and I'm very proud to have been invited to serve in his government.
"It's not what I was seeking and it's certainly not what I was expecting but nonetheless it's a great opportunity and a great challenge and I look forward to returning to being with my colleagues and my friends again in the Labour government. "
He said everyone had to play their part in dealing with the global financial crisis, adding: "In a sense it's all hands on deck."
He joked "that's quite enough" when the BBC's James Landale suggested he was the "comeback kid".
Mr Mandelson is to be replaced as EU trade commissioner by Baroness Ashton, the leader of the House of Lords.
He served as trade and industry secretary and Northern Ireland secretary in Tony Blair's government but had to resign twice and his critics within the Labour Party say he remains a divisive figure.
He also had a long-standing feud with Mr Brown, after backing Tony Blair for the party leadership in 1994.
The decision to bring him back from Brussels has been warmly welcomed by some senior Labour figures, including former home secretary David Blunkett who described it as a "masterstroke". But others, including Labour backbenchers Ian Gibson, John McDonnell and Peter Kilfoyle have said they are very disappointed.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.