A Gallup poll of more than 2,000 adults found that nearly 42% had suffered from back pain.
Of these, nearly half (47%) said their problems were a result of working in the garden.
Among the 35-plus age group the proportion (56%) was even higher.
More than a third of these believed that the best way to deal with back pain is to lie down and rest.
This is in complete contradiction to medical advice, which is to take adequate pain relief and stay active.
Six out of 10 of those who suffered from back pain said the problem was so bad that it had stopped them taking part in their normal leisure activities.
The Gallup survey shows that many part-time gardeners are not aware of how to lift properly, how to avoid back strain and what to do if it occurs.
Gardening can cause muscle strains and sprains, and lead to spasms in the back.
Younger people are also at risk of rupturing discs in their back by lifting heavy weights.
The Ground Force effect
Dr Peter Skew, of the charity Back Care, said many people had been inspired to take up gardening after watching television programmes such as Ground Force.
"People watch Charlie Dimmock doing her bits and pieces and they get inspired to have the same sort of thing in their garden."
He said gardening was excellent exercise - provided people were sensible and took time to prepare properly.
He told BBC News Online: "Gardening is one of the best forms of exercise any of us can take, it is good for flexibility, strength and stamina.
"But you have to plan what you are going to do, and you have to pace yourself properly - don't try to do everything in one weekend."
Dr Skew said it was important to do stretching exercises before gardening.
He also suggested that new gardeners should do 30-40 sit ups every day for one to two weeks days before taking up the hobby. This helps to strengthen the stomach muscles, and stabilise the back.
Back pain is the UK's leading cause of disability affecting 1.1 million people.
It is estimated to cause the loss of 119 million working days annually - at a total cost of almost £4bn a year to employers.
The government has announced a Back to Work programme, which is part of the Healthy Workplace Initiative, aiming to encourage companies to adopt "back-friendly" policies.