By Paul Lewis
BBC Radio 4's Money Box
Tax changes to discourage people buying property to let it out have been called for by a housing body.
Many buyers feel buy-to-let landlords have an unfair advantage
The Chartered Institute of Housing says there is a crisis in affordable housing in the UK.
It blames the tax relief given to buy-to-let landlords on the interest paid on their mortgage.
Landlords say the crisis is caused by a shortage of new homes and ending tax relief would force many landlords out of business.
Paul Diggory, president of the Chartered Institute of Housing, told BBC Radio 4's Money Box programme: "Our main concern is the acute lack of affordable housing across the UK.
"We've got many examples now in many cities where properties are being secured on buy-to-let mortgages but are being kept empty."
Although empty property does not get any tax relief, when a property is rented out the cost of the interest on the mortgage can be deducted from the rent before tax is calculated.
That effectively gives tax relief on much of the cost of the mortgage, costing taxpayers an estimated £2bn a year.
Around 175,000 of the 1.3m homes sold with mortgages in 2006 went to buy-to-let landlords.
Paul Diggory said: "Buy-to-let owners [have] a financial advantage over those trying to buy their first home, pushing prices even higher - further out of reach. Why does the government still offer tax incentives to those who buy simply to rent?"
But the National Association of Landlords told the programme that any change in the tax situation would be disastrous.
We provide the homes that neither the state nor owner-occupation offers
John Socha, National Association of Landlords
Vice chairman John Socha told Money Box that buy-to-let purchasers have to find much bigger deposits than owner-occupiers and pay capital gains tax when they sell the property.
"Increasing the tax will squeeze out a lot of landlords, most of whom are doing it for their pensions.
"The government has said it wants a healthy private rented sector. We provide the homes that neither the state nor owner-occupation offers."
But he said he shared Paul Diggory's concerns about empty homes.
"We don't agree with people keeping properties empty. I don't think anyone thinks that's a rational thing to do."
But Paul Diggory said that he wants empty properties taxed as well.
"We would like to see some sort of new tax on homeowners who keep properties empty. We have to tackle empty homes which are such a waste of a vital resource."
BBC Radio 4's Money Box was broadcast on Saturday, 23 June 2007 at 1204 BST.
The programme will be repeated on Sunday, 24 June 2007 at 2102 BST.