A decision to bug a conversation between a Labour MP and his constituent was taken by a Thames Valley Police officer, the BBC understands.
Sadiq Khan has welcomed Justice Secretary Jack Straw's inquiry
It follows claims counter-terrorism officers secretly recorded discussions between Tooting MP Sadiq Khan and a constituent he was visiting in jail.
Jack Straw told MPs ministers were not consulted about the alleged bugging.
Chief Surveillance Commissioner Sir Christopher Rose is to head an inquiry into the allegations, Mr Straw said.
Justice Secretary Mr Straw told MPs the former Court of Appeal judge would try to find out under whose authority any bugging was carried out.
He said Sir Christopher would try to complete his task within two weeks and would report back the results to the prime minister, home secretary and Mr Straw.
He insisted that a chief police officer had to authorise eavesdropping operations, adding that "ministers play no part in these authorisations".
"This is the first time I can recall such an allegation being made," he said.
Committee to investigate
But shadow home secretary David Davis said the case had exposed critical failings in the system.
"First that it's possible for the executive to ride roughshod over the relationship between a member of Parliament and his constituents, the very basis of parliamentary democracy," he said.
"Second that the necessary authorisations for secret anti-terrorist activity may be being ignored."
BBC Political Editor Nick Robinson said he understood the decision to bug Mr Khan "was taken by Thames Valley Police", but that the MP "was not the target of the bugging".
He said a police officer, who is facing disciplinary action on an unrelated matter, was involved but would claim he had his chief constable's approval.
The Commons home affairs select committee is also to investigate the allegations as part of its inquiry into the "surveillance society"
The conversations alleged to have been recorded took place in 2005 and 2006 at Woodhill Prison, in Milton Keynes, Buckinghamshire.
They were between Mr Khan and Babar Ahmad - a constituent and childhood friend - who is in prison awaiting extradition to the US.
He is accused there of running websites supporting the Taleban and Chechen terrorists, though he faces no charges in the UK.
Jack Straw says recording MPs' conversations is "unacceptable"
Mr Khan, a government whip, has campaigned for Mr Ahmad's release.
Police are entitled to monitor prisoners' communications but such operations must first be approved by the surveillance commissioner who considers them on a case-by-case basis.
However, a code known as the Wilson Doctrine forbids the covert recording of conversations between MPs and their constituents.
During the Commons debate earlier David Heath, for the Liberal Democrats, said: "Does the fact that protocols are apparently not understood by senior police officers not undermine the safeguards offered and enacted on the ever-expanding surveillance activity in this country?"
Mr Khan has welcomed the probe, saying that if his conversation was recorded, it would undermine the basis of the relationship between MPs and constituents.
Scotland Yard has refused to comment on the allegations, made in the Sunday Times.
Mr Straw has said he has no knowledge of what had happened but that it is "completely unacceptable" for MPs' conversations to be recorded.
The Conservatives have published a letter which they say Mr Davis sent to Prime Minister Gordon Brown in December alerting him to the possible breach of the doctrine.
But Downing Street says it has no record of the letter being received by Mr Brown, despite "a detailed check" and the prime minister knew nothing about it.
Conservative leader David Cameron said: "They lost 25 million tax details - I suppose one letter pales in significance."
He said "some very clear answers" were needed from the government about "who knew about the bugging" and if they knew they were effectively bugging an MP "did they ensure it was authorised at the highest level".
HAVE YOUR SAY
MPs are members of the public and should be subject to the same laws as the rest of us
Liberty director Shami Chakrabarti said: "It's high time for simpler and stronger surveillance laws with warrants issued by judges, not policeman nor politicians."
Meanwhile human rights lawyer Imran Khan, who represents several terrorism suspects, has written to Mr Straw asking that the inquiry also consider whether solicitors conversations with clients were routinely monitored.
He told the BBC inmates at Belmarsh and Woodhill prisons felt they were, adding: "If it's routine and it's simply about intelligence-gathering then it really undermines the principle of confidentiality in the justice system in this country."