Official advice is that pregnant women can safely drink up to four units of alcohol a week.
But researchers at Queen's University have said their tests on babies in the womb have revealed that even drinking the equivalent of four glasses of wine a week can affect development.
The government is being urged to re-consider its advice in light of the new findings.
The tests were carried out on over 100 women 25 weeks into their pregnancy.
At that stage, most babies jump as if startled when a buzzer is sounded on their mother's abdomen.
This response is interpreted as healthy by doctors as it indicates that the brain and central nervous system is functioning properly.
However, the researchers said that even among women drinking four units of alcohol a week, significantly fewer babies passed the startle test than those whose mothers drank nothing at all.
Central nervous system
Research fellow Dr Jennifer Little, who conducted the study, said: "The results show that even low levels of alcohol may have an effect on the central nervous system functioning.
"We don't want to concern women, but until we can absolutely say that a certain level of alcohol will have no effect, I would urge caution."
American women are advised not to drink at all during pregnancy but more than two-thirds of British mothers-to-be do not totally give up.
The Royal College of Midwives had previously said up to eight units was a safe limit.
A spokeswoman said: "The research is changing all the time and we do not want to alarm women who have the occasional drink.
"Light drinking is not harmful but the only way to guarantee your baby won't be affected is not to drink at all."
Excessive alcohol abuse during pregnancy can ultimately lead to Foetal Alcohol Syndrome.
A spokeswoman for the Department of Health said: "Our advice was based on evidence from the expert committee on toxicity in 1995.
"This research was limited to 120 babies. The overwhelming body of evidence suggests that moderate drinking is not harmful, and most pregnant women drink below the levels we advise."