One in six councils is failing to protect vulnerable adults in their care, the BBC has discovered.
Abused adults need the same protection as children say campaigners
Even some councils which received the highest government rating for their social services have fallen short in their protection of vulnerable adults.
The government's director of inspection Mike Rourke told BBC File On 4 these adults need similar legal safeguards to child protection law.
Minister Ivan Lewis said he is looking to strengthen adult protection.
File On 4 requested a detailed assessment of adult protection from the Commission for Social Care Inspection and found that a sixth of all 150 councils in England were failing.
Confirming the BBC's analysis as a "serious" problem, Mr Rourke, the CSCI's Director of Inspection, said: "Our message is that councils have not got systems as tight as they should be and therefore cannot be sure they are responding adequately to referrals - that's our challenge to them."
He said there were various reasons for the failure. "For some it's financial priorities, others it is competing priorities and for some it's a lack of management grip," he said.
Disability campaign group Voice UK said the figures prove that "No Secrets", a government policy set out in 2000 to stop the abuse and neglect of elderly or disabled people, has been poorly implemented.
Kathryn Stone, Voice UK chief executive, "It might be argued it is not effective because there is no law to support it - we have campaigned for adult protection legislation to raise it to the same status as child protection."
One case cited by the group is that of "James", whose family allege he was raped and sexually abused by another resident while in sheltered accommodation.
The family also say £2,000 worth of property was stolen from him.
An independent review "largely upheld" the family's complaints against Halton Council, in Cheshire, but the James's relatives said they have not received an apology and a satisfactory explanation from the council.
Halton Council refused to comment.
Voice UK said the case illustrates a national pattern where families who allege relatives have been abused are seen as troublesome by local authorities.
Campaigners for the elderly are also concerned at the lack of protection for elderly people.
Ivan Lewis said he is looking to strengthen adult protection
Gary Fitzgerald, Chief Executive of Action on Elder Abuse, called on Care Services Minister Ivan Lewis to honour a pledge made by a previous minister to make all official agencies safeguard people at risk from abuse by setting tough targets in their appraisals.
"We are still waiting for progress," he said.
Mr Lewis told the BBC it was unacceptable that one in six councils were failing to protect vulnerable adults.
He said the government and the charity Comic Relief are spending £650,000 to research the level of abuse on older people which is due to be published in the summer.
"We will decide then what action to take as a government and a society," Mr Lewis added.
Hear the full story on Radio 4: File on 4 Tuesday, 20 March, 2007 at 2000 GMT.