Two housing bodies have said Wales is short of about 40,000 affordable homes.
The Home Builders Federation and the Chartered Institute of Housing Cymru said fewer new homes were being built in Wales than at any time since WWII.
They criticised the assembly government and local authorities over the planning process and house-building policies.
The Welsh Assembly Government said it had raised spending on social housing. The Welsh Local Government Association called their views "simplistic".
The two housing bodies blamed rocketing house prices for making life difficult for many first-time buyers.
They said they had identified a chronic shortage of affordable homes and claimed that affordable housing - which are part-owned or with subsidies available to buyers - was the only option for many people.
The two organisations said the planning system was too time-consuming and that not enough land was made available for development.
Richard Price, a planning and policy advisor with the Home Builders Federation for Wales, said: "It seems bleak at the moment, we've got a very big affordable housing crisis.
"Considering the fact that the areas that are most attractive to investment are planning a reduction in house-building doesn't really do us any favours.
"And it just looks as if it's going to get worse unless we really do something to tackle these problems."
Eight people queued overnight for the sale of these ex-military homes
Kellie Beirne, of the Chartered Institute of Housing Cymru, said the problems were across the board but the assembly government and local authorities had a part to play in finding a solution.
"The population is increasing and more households are forming, but the average size of these households is declining," she said.
In response, Councillor Aled Roberts, the Welsh Local Government Association's spokesperson for housing, said: "What they are saying about affordable housing is simplistic.
"Some developers claim that the imposition of affordable homes is detrimental to them.
"The Town and Country Planning Act is there so that there is no indiscriminate building.
"We have to find a balance between those who want to buy their own properties and those communities who would be aghast at a complete loosening of planning guidelines.
"We need to look at the supply of housing. There is an issue with the number of empty properties which need to be brought back into circulation.
"There is the issue of pressure on the system caused by the buy-to-let sector."
In a statement, the Welsh Assembly Government said it had increased investment in the social housing by 72% since 2004/05 and planned to invest a further £450m.
Social Justice Minister Andrew Davies said it was "a very complex issue". "It is partly to do with the way the market has worked, availability of land and many other issues but clearly affordable housing is one of the key issues facing us in Wales," he said.
"We have very considerably increased investment and will be working with both public sector and the voluntary sector such as housing associations, as well as the private sector. We need to work together with the house builders."
Eight people have queued overnight to prepare for the sale of 14 former Ministry of Defence homes at Annington Green, on Deeside.
The homes, which at £112,950 which are sold on a first-come, first-served basis, are secured with a down payment of £250.