Older people are moving into villages at the same time as the young leave
A quarter of people living in England's countryside will be over 65 by 2020, a campaign group has said.
The National Housing Federation said figures estimate a 40% rise in older residents over the next 10 years.
Communities will struggle to support the ageing population in villages unless more affordable homes are built, it said.
The Department for Communities and Local Government said it was committed to delivering more affordable homes.
It was investing £7.5bn over the next two years, it said.
The federation, which represents England's housing associations, said the number of over 65s living in rural England is expected to be 3.23m by 2020, compared with 2.32m in 2008.
The rising cost of housing in rural areas is driving out younger people as houses are, on average, £40,000 more expensive than those in towns and cities - even though wages are far lower, it said.
At the same time, better-off older people are retiring to their dream country home.
The federation said this meant many of the rural elderly would find themselves isolated, with fewer young neighbours to offer a helping hand.
Rural local authorities need to assess what housing and support services will be needed to serve the growing older population
National Housing Federation
It predicted this would put huge pressure on an already stretched public purse, as social services and other agencies would have to step into the breach.
The federation called for more affordable housing to be built for younger people in rural areas, to help retain a cross-section of age groups and keep local schools, bus services, shops and pubs open.
It estimated about 100,000 new affordable homes needed to be built in England to meet demand in rural areas over the next 10 years.
Young people would also need to be recruited to meet an increased demand for health care workers in the countryside, it said.
More "pensioner friendly homes" would also needed.
The federation estimates one in three older people in rural areas is expected to live alone by 2020, compared with one in four in 2010.
Federation chief executive David Orr said the rising number of older people living in villages would bring "benefits" but also "a series of big challenges".
"If families and younger people are priced out of the countryside and local services and amenities continue to disappear, older people will find themselves increasingly isolated.
"That's why rural local authorities must carry out assessments of housing needs every three years and draw up action plans. They also need to assess what housing and support services will be needed to serve the growing older population," he said.
A Communities and Local Government spokeswoman said it was providing funding to housing associations to build more affordable homes for sale and rent, and had invested in the largest council house-building programme in nearly two decades.
"The government has also introduced the Homebuy schemes to increase opportunities for people to get on the housing ladder, which more than 100,000 people have benefited from so far.
"In rural areas where replacing affordable housing is difficult, new shared ownership properties will have to remain shared ownership to ensure future buyers also have a chance to step on to the property ladder," she said.
Currently throughout the UK, 16% of the total population is aged over 65.