Some argue too many student houses result in anti-social behaviour
Landlords and the National Union of Students (NUS) have rejected government proposals to limit house-sharing.
The government may give councils the power to limit the number of houses in multiple occupation (HMOs) in one area.
Locals complained that too many HMOs lead to problems of anti-social behaviour as well as areas being left empty during holidays.
The plan's opponents say the powers would affect migrant workers and young professionals as well as students.
"It is extremely foolish to propose that we displace all of these people in the middle of a housing crisis," said Wes Streeting, president of the NUS.
A spokesman for Communities and Local Government said: "Students bring benefits to the places they live in, but too many residing in one area can impact negatively on a community".
"This is a real problem in many communities across England, which is why Communities Secretary John Denham is committed to finding a long term solution to current rental practices."
The idea would be to limit the number of houses in a single area in which more than six unrelated people are living.
The government is currently looking at the responses to its consultation and will decide whether to make any changes to the plans "in the near future".
In a report about so-called studentification last September, John Denham said, "Cities including Liverpool, Loughborough, Leicester, Nottingham, Southampton and Bristol have reported more empty properties during the summer meaning shops, businesses, community facilities and pubs simply close down creating 'ghost towns'."
"In addition there can be anti-social behaviour, litter and parking problems during term time."
The NUS, together with the British Property Federation, Federation of Small Businesses, National Landlords Association and Residential Landlords Association, has sent a letter to Mr Denham.
The letter describes the proposals as discriminatory, and says they "should not be enacted to control what is a problem in only a very small number of wards around the country".
Alan Ward, chairman of the Residential Landlords Association also pointed out that the proposals could cause problems for mortgage-holders.
"Planning permission can affect the marketability and value of property, forcing landlords to repay capital on their loans, or sell, resulting in loss of homes for tenants."