Dieting mothers do not give their babies the types of food they need to grow, research suggests.
The researchers looked at what babies were being fed
Some mothers even worry that their baby could be "overweight".
Babies need to have fat and carbohydrates in their diets while they are developing.
But a survey of over 700 mothers found just 3% thought fat was the most important element in a baby's diet, and 2% highlighted the need for carbohydrates.
The survey found 43% believed vitamins and minerals were most important in their baby's diet, 17% said iron and 15% calcium.
It suggested that, contrary to government advice, 25% of mothers give their children cow's milk as their main drink before they are a year old.
A quarter give their babies solid food before they are three months old, which is also earlier than recommended.
Weaning before four months is not recommended because the baby's digestive system is not developed enough to cope with solids.
Giving cows' milk as a main drink before 12 months of age has been linked with an increased risk of developing iron deficiency and anaemia.
The survey was conducted via Mother and Baby magazine in March this year for a leading baby food manufacturer.
Luci Daniels, a state registered dietician, said: "An increasing number of mums worry that their baby is or could become overweight and are feeding them low fat foods and low calorie meals that are unsuitable.
"The research suggests that some mums are using their knowledge of adult weight loss and `healthy eating' when choosing what their babies eat, without understanding that it could be damaging for their child."
But she warned if babies and toddlers did not get the energy they needed, their growth and development could be affected.
"A healthy weaning diet is not the same as a healthy adult diet."
Ms Daniels added: "The majority of mums are getting it right.
"However the fact that as many as one in four are introducing solids and a
similar number are making the switch to cows' milk too early is a cause of concern."
Amanda Wynne of the British Dietetic Association told BBC News Online: "It is a matter of concern when mothers are perhaps a bit overzealous over the diets of their children.
"Children under five shouldn't be on healthy eating, low-fat high-fibre diets."
Ms Wynne added: "You shouldn't wean children to early. It's best to do that when they are four to six months old."
She said the best drink for young babies to be given was breast milk.
Once children begin drinking cow's milk, they should be given full-fat milk until they are five.
Rosie Dodds, policy research officer for the National Childbirth Trust, said she believed there were some mothers who were concerned about their own fat intake who were also worried about their children's.
But she said: "I don't think this is a big problem because babies are getting most of their nourishment from breastmilk or formula and it is relatively rare to see a child with faltering growth."
Ms Dodds added: "Starting babies on solids too early has been shown to increase a baby's risk of eczema and allergy.
"They are also at a greater risk of gastroenteritis . Their guts aren't really designed to have solid foods until they are around six months old."
But she suggested the risks to children of giving drinking cow's milk before they were a year old could have been exaggerated.
Babies can have cow's milk with foods such as mashed vegetables or porridge, she said.
"There is some evidence that children are more likely to be anaemic on cow's milk, but the research tends to be funded by formula milk manufacturers."