Experts at the World Health Organisation's International Programme on Chemical Safety say that research linking this type of pollution to health problems is patchy.
However, it says that the known effects on wildlife are "extensive", and that more studies are needed to pinpoint the danger to humans.
Endocrine disrupting chemicals (EDCs) often have the same properties as hormones that the body uses to control a wide variety of functions, including reproduction.
People may be widely exposed to them through pesticide residues remaining on food, plastics, household products and industrial chemicals.
Many experts are worried that exposure may be interfering with fertility - or even accelerating the growth of certain cancers.
However, apart from evidence linking high exposure levels with health problems, the evidence against them is inconclusive, says the report.
"The state-of-the-science assessment has revealed that our current understanding of the effects posed by EDCs to wildlife and humans is incomplete," it says.
"The evidence that high-level exposure may impact both humans and wildlife indicates that this potential mechanism of toxicity warrants our attention."
Health risks
In humans, EDCs have been linked to a number of health problems.
These include:
A number of studies have reported a decline in human sperm quality - however, there is no firm link to EDC exposure.
Animal threat
In wildlife, problems have been noticed in many species.
Examples include population decline among Baltic seals due to exposure to pesticides such as PCBs and DDE, and eggshell thinning and altered gonadal development in birds of prey exposed to DDT.
A pesticide spill in a Florida lake caused a sharp population decline among alligators, while there are suspicions that fish are suffering because of EDCs in effluent.
Campaigners say that the evidence for potential harm to humans is already strong enough to support tough measures.
Gwynne Lyons, from WWF, said: "We cannot wait decades for precise causal mechanisms to be established.
"We owe it to our children and to wildlife to act now to eliminate exposure to man made hormone disrupting chemicals."
'Major health issues'
Dr Richard Sharpe, from the UK Society for Endocrinology, said:"It is becoming clearer that many of the major health issues that face humans in Western developed countries are associated with hormonal problems.
"Against this background, there is understandable concern that human exposure to certain environmental chemicals might cause or worsen conditions that result from altered hormonal action."