Tamils want the British government to help stop fighting in Sri Lanka
Commons Speaker Michael Martin has accused hundreds of Tamil protesters of "hijacking" Parliament Square.
Demonstrators have gathered in the square since the beginning of April, causing disruption to traffic.
Mr Martin said the protests were preventing others from "exercising their democratic duty to demonstrate".
Following complaints from several MPs about the disruption, he said he would hold a meeting in the Commons for those involved in a bid to resolve the issue.
The demonstrators are angry at the impact of recent fighting in Sri Lanka on civilians.
They claim that human rights abuses are taking place and want the British government to intervene and push for an end to the conflict.
But shadow defence minister Gerald Howarth said: "It surely is quite unacceptable that these people should be allowed to take over Parliament Square, disrupt the entire centre of our capital city and I wonder what on earth the Metropolitan Police Commissioner's doing about it."
Mr Martin replied: "I have a great deal of sympathy with the case that you make.
"Many of us before we came into the House were involved in demonstrations, because it is part of a democracy. We have a demonstration and leave.
"No-one has ever expected a demonstration to be such that people have hijacked Parliament Square and the roads, and therefore stopped others from exercising their democratic duty to demonstrate."
Labour MP Jeremy Corbyn asked for reassurance that the Tamils' right to demonstrate would not be restricted.
"Could we all not have some sympathy for the Tamil people out there who are just desperate to do something to try to bring about safety for their families back home," he said.
But Mr Martin said that while the demonstrators had "issues that they want to be heard," they had overstepped the permission they had been given to make their case in Parliament Square.
"I know I'm in a bit of a bad mood today, but let me say this - when authorisation is given for a demonstration to cover 50 people, that means 50 people," he told the House.
"It doesn't mean tents going on, it doesn't mean food... that's not authorisation for a demonstration."