They question whether the city can cope with the influx.
More than 7,000 asylum seekers and their families are expected to be housed in Springburn and Sighthill over the next five years.
A deal with the UK Government means the city stands to gain £100m for housing the families in some of Glasgow's 2,000 vacant homes.
However, critics say placing them in deprived areas only exacerbates existing social and economic problems.
They question whether the city can cope with the influx.
Speaking on BBC Frontline Scotland, to be broadcast on Tuesday evening, Springburn GP Dr Iain Brown said: "You have to recognise that the reason the accommodation is there in the first place is that the area has problems.
"To put this additional burden on to this area without trying to sort out some of the underlying problems is folly."
A spokesman for the council said the authority did not seek to defend the conditions in which far too many council tenants live, but it did not believe that was a reason for refusing to help asylum seekers.
David Comley, Glasgow council's director of housing, said: "It is quite clear the asylum seekers are not in a position to furnich their accommodation and we have to provide furnished accommodation.
"The rents that are paid to us by the Home Office reflect the fact that we are letting furnished, so the council won't make a profit out of this."
Tax burden
Money to fund the asylum seekers' stay does not come directly from council tax payers.
Many towns in the south of England have been struggling to cope with the number of asylum seekers, prompting the Home Office to offer incentives to other cities to help out.
Attempts by Glasgow City Council to provide accommodation have so far been beset by problems.
Police restrictions
A group of Romanian families who arrived in March were sent back to London after it was discovered they had been ordered to report to Wandsworth police on a regular basis.
Following that, there were reports asylum seekers were reluctant to travel to Glasgow.
There have also been concerns that voucher schemes put in place for the families were too restrictive.
Many of the asylum seekers have been housed in areas like Sighthill, where relations with locals are a little frosty.
Charlie Riddle of the Sighthill Tenants Association said: "It's not the presence of the people themselves - we're used to people from all over round here.
"It was this business of the fact that everybody round here is very poor and they don't have the kind of things these people seemed to have been given."