He denied he had joined the government, saying: "I wouldn't join the government, I don't see this as a political thing."
He said he made it "perfectly clear" to Mr Brown that he would not compromise his position on The Apprentice - the BBC show that Sir Alan fronts.
He said he had spoken to the BBC in advance for guidelines on the issue adding: "It's very simple - all I am is an adviser, I'm not a policymaker."
Asked whether he would become a peer, Sir Alan replied: "That's what I hear."
Asked whether he would continue in the "enterprise tsar" role under a Conservative government, Sir Alan - who said he had known Mr Brown for years - said: "As far as I'm concerned this is politically neutral. All I want to do is try and help business and entrepreneurs."
On the Tory leader he said: "I have never met Mr Cameron and I don't know anything about him."
But the Conservatives said Sir Alan's role on The Apprentice was now "completely incompatible" with BBC rules.
Shadow culture secretary Jeremy Hunt said: "Presenting a programme for the BBC and working for the government on the same issue is totally incompatible with the BBC's rules on political independence and impartiality.
"Sir Alan Sugar needs to make a choice between his role in The Apprentice and his role as the government's business tsar."
He said he had written to BBC Trust chairman Sir Michael Lyons to ask "as a matter of urgency to explain who at the BBC gave guidance to Sir Alan and whether he had informed them that he would be a Labour peer".
A spokesman for the BBC Trust said it was aware that Mr Hunt had written to its chairman and would respond shortly.
A BBC spokesman said: "Sir Alan is in discussions with us about his plans and has assured us that he is determined not to do anything that would jeopardise his work at the BBC, which is something he greatly values."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.