Tony Blair and Lord Birt have been criticised by MPs over the peer's refusal to answer to Parliament.
Lord Birt has declined to appear before a committee before
The former BBC director general is employed as a "blue skies thinker" in the prime minister's strategy unit.
But a Commons committee has accused him of hampering its probe into strategic thinking and planning in government by refusing to answer its questions.
There was "resistance" from Number 10 for advisers to appear despite earlier assurances they would do so, it said.
The BBC's Sean Curran said the public administration select committee had wanted to know how Mr Blair identifies issues he wants to concentrate on and what happens as a result.
However, the Cabinet Office, on receiving the request for Lord Birt to appear, said the prime minister felt Stephen Aldridge, acting head of the strategy unit, was "best placed" to answer MPs' questions.
Committee chairman Tony Wright then wrote to Lord Birt suggesting he was "ideally placed" to help the inquiry.
The peer had conducted a newspaper interview and an Edinburgh Television Festival lecture, he noted.
"We would very much value your insight on how good strategy is made," the Labour MP wrote.
"I hope, therefore, we can now look forward to receiving a prompt, and positive, reply to our invitation."
However Lord Birt, who had refused to appear before a transport committee two years ago despite being the prime minister's transport adviser at the time, declined.
He replied: "As the acting director of the strategy unit, Stephen is ideally placed to explain how strategic thinking is undertaken at the centre of government."
The committee's report said: "This is hampering, unnecessarily, the ability of Parliament to undertake effective scrutiny.
"It is clear that there is strong resistance to the appearance of special advisers from Number 10 Downing Street.
"We are disappointed that the prime minister appears to feel that there is no presumption that his own key advisers will appear before select committees when they are requested to do so, notwithstanding the general undertakings that have recently been given to select committees on this matter."
Cabinet minister Peter Hain has previously said there is a "presumption" that special advisers would give evidence to committees if asked.
This assurance came after the prime minister had promised to see what could be done.