The White House has denied US media reports that the president had lost confidence in the CIA chief, after his surprise resignation on Friday.
Porter Goss's resignation surprised friends and pundits alike
Porter Goss, who was given the job of reforming the agency after a series of intelligence failures, served in the role for less than two years.
A White House spokeswoman said there was a "collective agreement" the CIA needed a new leader now.
However, Mr Goss said the reason for his departure would stay a "mystery".
Speaking on Saturday, Mr Goss declined to comment on his departure, telling CNN that "it's one of those mysteries."
The US media was full of speculation about Mr Goss' departure.
Graduated from Yale, 1960
Speaks Spanish and French
Was an undercover CIA spy for 10 years until a near-fatal illness forced his retirement in 1971
Elected to Congress in 1988
The Washington Post cited senior administration officials as saying President George W Bush had lost confidence in Mr Goss and had decided to replace him months ago.
White House spokeswoman Dana Perino called such reports "categorically untrue".
"There was a collective agreement that now would be a time that we could have a new CIA director come in and take the ball and move the agency forward," she said.
A BBC Washington correspondent says the most likely reason for his departure is that Mr Goss objected to his boss, John Negroponte.
Mr Negroponte was appointed to the post of national intelligence director last year.
Correspondents say that a replacement for Mr Goss could be announced as early as Monday, with Air Force General Michael Hayden - top deputy to Mr Negroponte - widely tipped to take over.