A third of people in the South East are dissatisfied with call centres run by the government and utility firms.
The study said 80% of people in the South East used call centres
A report said complaints included being left on hold for too long and dealing with unfriendly and unhelpful staff.
The Citizens Advice Bureaux (CAB) research found 27% of people were dissatisfied with finance companies' call centres.
CAB chief executive David Harker said call centres should not be a "barrier" to gaining information.
Critical debt problems
"Almost all call centre users have criticisms and a significant proportion are left dissatisfied," he said.
"Critical problems about money, debts and benefits, often affecting people on very low incomes, simply are not getting resolved as quickly as they should be.
Hanging on the Telephone - statistics for the South East
98% were annoyed by at least one aspect of the service
45% said they were left on hold for too long
36% were dissatisfied with centres run by utility companies and government agencies
27% were dissatisfied with finance institutions' call centres
26% were annoyed by unfriendly or unhelpful staff
"Call centres should be an effective gateway to services and problem resolution - not a barrier."
He said a Citizens Advice Bureau in Surrey contacted a retailer on behalf of an elderly person.
The company did not answer the phone and a series of automated options were given before the line went dead.
And in Kent, a single mother who was told she had been overpaid £1,900 in tax credits telephoned a call centre who could not give her any information.
He said the public could log on to the charity's website to share their worst call centre experiences.
Call centre managers are being asked to run regular audits to on their performance, accessibility and friendliness, he added.