There is not enough evidence to support extending the 28-day limit on holding terrorism suspects before charge, a committee of MPs and peers has said.
Gordon Brown wants to double detention to 56 days
Any extension should be based on clear evidence, not the idea that a need "may arise at some time in the future", said the joint committee on human rights.
Ministers have suggested doubling the period to 56 days saying more time may be needed for more complex cases.
The Tories and Lib Dems say they have not seen the evidence to justify it.
The Home Office has said Prime Minister Gordon Brown wants a political consensus on the issue.
Home Secretary Jacqui Smith has said there is a "weight of opinion" that "we may well be getting to the point where 28 days will not be long enough" to be in a position to charge someone, due to the growing complexity of some terrorist plots.
The committee's Labour chairman, Andrew Dismore, admitted the government had a "difficult job" in protecting the public while avoiding creating bad feeling and alienating Muslim communities.
But he said there was only one serious alleged plot where six people were held up to 28 days - three of whom were then released.
"You could say, on the one hand, those three, there may have been evidence against them, but equally, were they simply being held in case something turned up?," he said.
He said the police were not asking for an extension yet, and even Mr Brown had been "very cautious" in saying there may be circumstances in which it would be necessary to hold a suspect beyond 28 days.
The cross-party committee's report concluded that the law should only be changed if it was "justified by clear evidence that the need for such a power already exists".
'Impact on liberty'
It should not be altered because of "precautionary arguments that such a need may arise at some time in the future", it said.
Critics say extending the detention period further amounts to internment and fuels anger among some Muslims. The committee said it could have "a significant impact on liberty".
It also backed a full adversarial hearing before a judge when deciding if a suspect needed to be detained for longer without charge.
Currently, this can be extended in the detainee's absence or can be based on information not available to them.
This was "very far removed from anything we would consider to be a fair procedure", the report said.
The committee also recommended the production of an annual report on police use of the power to detain without charge beyond 14 days, and for a parliamentary-approved upper limit on pre-charge detention.
The government had originally wanted police to be able to hold terror suspects for 90 days without charge - but was defeated in 2005 by a combination of Tory, Lib Dem and some Labour MPs.
A compromise eventually saw the previous 14-day limit extended to 28 days, but a number of ministers have said they still favour a longer detention period.
"The government will be discussing this with colleagues and hopes to achieve a consensus before coming to a decision," a Home Office spokesman said.
"The police believe it is right and proper for the government to address this issue and they want us to discuss it with colleagues in government and more widely, which is why we are consulting."