The children who could benefit most from out of school clubs are least likely to have access to them, according to a report.
Children from affluent families take activities like swimming for granted
Young people on free school meals were less likely to participate in after school activities than those from more affluent homes, research suggested.
This was because rich parents were able to buy their children access to such clubs, while poorer parents could not.
The claim came in a guide for potential donors by New Philanthropy Capital.
The report by the charity, which advises donors on giving, said such out of school activities could play a key role in developing young people's skills and resilience.
Recognising this, the government has provided significant funds to help schools extend their services and provide more out of hours activities since 2002.
But funding is "extremely fragmented" as there are very few sources of cash specifically ear-marked for them.
Report author Clare Yeowart said: "It's probably the kids who don't get much support at home who need activity programmes the most.
"Yet struggling schools in disadvantaged areas often lack the resources to offer them."
She says such activities are often run by volunteers and rely on the willingness of teachers to give up their time.
"Schools have so many demands on their budgets and on staff time that out of school activities may simply get freezed out, particularly as government funding is not ring-fenced for out-of-school activities and is not enough to keep them going," she added.
Some clubs are open to everyone and some target a specific group such as underachieving or gifted and talented children.
Some charge a small fee to help cover running costs or because they believe it encourages a sense of value in the club.
But these are often waived in cases where they would be prohibitive.
Other barriers, such as transport or negative perceptions of the school in which they may be held, can also prevent youngsters from attending however.
A Department for Children, Schools and Families spokesman said the government agreed that extra-curricular activities support children’s educational and social development.
This was why it had just announced £160m to improve the quality and range of places for children and young people to go and things for them to do, he said.