Ms Harman said MPs had to be careful when talking about the police probe
The Conservatives and Liberal Democrats have said they will not take part in a Commons inquiry into the police raid on MP Damian Green's parliamentary office.
The government won a close vote to set up a committee dominated by Labour MPs, which would not begin work until after the police investigation finished.
The Lib Dems denounced the inquiry as a "nonsense", while the Tories warned it should not be "party political".
A cross-party motion to change the committee's remit failed by four votes.
Suggested by former Lib Dem leader Sir Menzies Campbell, the amendment would have allowed the inquiry to go ahead without waiting for the completion of any police inquiry or proceedings.
After this failed, a government motion to set up an MPs' committee in accordance with the make-up of the Commons - which has a Labour majority - and which will not start work until the police inquiry is over, passed by 23 votes.
'Should meet now'
For the Tories, shadow Commons leader Theresa May said: "We feel not able to recommend to members of our party that they serve on this committee which we believe so blatantly flies in the face of the desire that you had about the nature of the committee, the fact that it should not be party political, and that it should meet now and conduct its inquiries now."
Lib Dem business spokesman Simon Hughes said: "We do not feel we can recommend to our colleagues that they serve on a committee that will be a nonsense, that sits to choose a chairman and then can sit no more."
He added: "The government has used its majority to castrate Parliament and prevent proper parliamentary investigation into the police raid on an MP's office."
Many MPs are concerned that police were allowed to search Mr Green's Westminster office 11 days ago without a warrant.
Commons leader Harriet Harman said the next House of Commons Commission meeting would consider whether a High Court judge should have to grant a warrant if police wanted to carry out a parliamentary search in future.
She told MPs the issues that needed to be considered were whether MPs' papers were "properly protected", what led up to the police search and whether "important constitutional principles were properly protected".
We all agree there's something called parliamentary privileges but hardly anyone agrees exactly what it amounts to
But she said it was important to avoid any perception that Parliament was "breathing down the police's neck" or could prejudice any future court proceedings.
"It is for neither government nor opposition to pronounce what the evidence is and whether a crime has been committed," she added.
The opposition parties are angry that, while Speaker Michael Martin promised a "speedy" inquiry by a committee of seven MPs chosen by him, the government has proposed a committee dominated by Labour MPs which will not start work until after the police investigation has finished.
During a lively Commons debate, Conservative former minister Douglas Hogg accused ministers of "concealment, duplicity, whitewash and cover-up".
Tory backbencher Sir Nicholas Winterton said that, even in Zimbabwe in 1982, Robert Mugabe's police had not entered Parliament to arrest an MP.
'Over the line'
However, Labour former minister Frank Dobson told MPs the rules on police entering the Commons were "not clear".
"We all agree there's something called parliamentary privileges but hardly anyone agrees exactly what it amounts to," he said.
Mr Green stood up at one point to caution Mr Dobson who suggested the Tory MP had a "sort of standing order with this civil servant" to get a continuation of leaks.
He said the Speaker had warned against saying anything prejudicial and that Mr Dobson had gone "way over the line".
Mr Green was arrested last month and held for nine hours by the Metropolitan Police.
His homes and offices were searched, as part of an inquiry into allegations of leaks from the Home Office.
Speaker Michael Martin has already said he regrets the fact that police were allowed to enter the Commons without a warrant.
But the Speaker himself did not come in for criticism during the debate, despite 32 MPs telling the BBC at the weekend they had lost confidence in him.
The Commons home affairs committee is expected to meet on Tuesday to discuss whether to hold an inquiry into the arrest of Mr Green and the search of his Commons office.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.