A bent X chromosome like the one above is characteristic of the condition
Delays at crucial points during the development of the brain in the womb may explain why people with a condition linked to autism do not like hugs.
A study in mice with fragile X syndrome found wiring in the part of the brain that responds to touch is formed late.
The findings may help explain why people with the condition are hypersensitive to physical contact, the researchers wrote in Neuron.
It also points to key stages when treatment could be most effective.
Fragile X syndrome is caused by a mutant gene in the X chromosome that interferes in the production of a protein called fragile X mental retardation protein (FMRP).
Under normal circumstances, the protein directs the formation of other proteins that build synapses in the brain.
Boys are usually more severely affected with the condition - which is the leading known cause of autism - because they have only one X chromosome.
In addition to mental impairment, hyperactivity, emotional and behavioural problems, anxiety and mood swings, people with fragile X also show what doctors call "tactile defensiveness", which means they do not make eye contact and do not like physical contact and are hypersensitive to touch and sound.
By recording electrical signals in the brains of mice, bred to mimic the condition, the researchers found that connections in the sensory cortex in the brain were late to mature.
This "mistiming" may trigger a domino effect and cause further problems with the correct wiring of the brain, they concluded.
The study also found these changes in the brain's connections occur much earlier than previously thought, midway through a baby's development in the womb.
And it suggests there are key "windows" when treatments for fragile X and autism could be most effective, they said.
Professor Peter Kind, who led the study at the University of Edinburgh, added: "We've learned these changes happen much earlier than previously thought, which gives valuable insight into when we should begin therapeutic intervention for people with these conditions.
"It also has implications for the treatment of autism since the changes in the brains of fragile X and autistic people are thought to significantly overlap."
Dr Gina Gómez de la Cuesta, from the National Autistic Society, said research into fragile X syndrome could help understanding of certain aspects of autism.
"Autism is common in people with fragile X syndrome, however there are many other causes of autism, most of which are not yet fully understood.
"Understanding how the brain works when a person has fragile X syndrome could help put some of the pieces together about what is happening in the brain when a person has autism, but it is not the whole story.
"Animal research can tell us a lot about genetics and the brain, but it is only a small part of the picture and further research would be required before we fully understand any links to autism."