HRT could cause deafness and reduce the brain's ability to process information, claim researchers.
HRT has been linked to poorer hearing
A team from the University of Rochester Medical Center in the US found women who had HRT were up to 30% more hard of hearing than other women.
They studied two groups of 32 women and measured their hearing in noisy areas and ability to process information.
The study surprised researchers, who thought hormone therapy would have the opposite effect on hearing.
Dr Robert Frisina, Professor of Otolaryngology at the University of Rochester Medical Center, and colleagues conducted tests on women aged between 60 and 86.
They matched each group for age and medical history as closely as possible.
The women were tested by being asked to decipher sentences while listening to someone talking in a noisy area.
The team found women who had undergone HRT suffered a particular type of age-related deafness called presbycusis.
Presbycusis is a condition where high pitched sounds are harder to hear, while deeper, lower sounds are magnified. For example, it would be easier to hear a man's voice than a woman's.
It can be linked to physical changes in the middle ear or to complex changes in the nerve pathways leading to the brain.
The researchers are unsure why hormone therapy would have this effect.
Unhealthy levels of oestrogen
Dr Frisina told BBC News Online one theory is that taking a hormone pill could increase oestrogen levels to an extent that is not good for the brain.
"Oestrogen directly affects the levels of sodium and potassium in the body, which are responsible for nerve cell function," he said.
This could affect the functioning of the nerve pathways leading to the brain, hindering the brain's ability to process information.
He said they originally believed HRT would improve hearing because the ear has oestrogen receptors.
Professor David Purdie, an expert on HRT from Edinburgh said he had great reservations about the study.
"The only thing that HRT has not yet been accused of is global warming and aggravating the national debt," he told BBC News Online.
"It's a small study and we've learned to exercise great caution in relation to US studies as they use a different type of HRT to what we use in the UK.
"Americans are notorious for using consistently higher doses of oestrogen."
He said oestrogen is a natural hormone and used cautiously and for the right reasons is usually safe.
Dr Frisina said a much larger study needs to be done before conclusions can be drawn.
This is the first time a study has linked hormone replacement therapy with deafness.
Previous research has linked HRT to breast cancer, Alzheimer's, asthma and heart disease.