But researchers from the Institute of Child Health found although both factors influenced test scores, social class had more of an effect.
This evidence of the importance of social background meant the problem of childhood deprivation had to be addressed, they said.
The team studied 10,845 men and women born in early March,1958, in England, Scotland, and Wales.
Maths scores
They examined the combined effect of birth weight and social class, based on the father's occupation.
Researchers looked at results from school tests, comparing results at ages seven,11 and 16 in maths, reading, general ability and perceptual and motor skills.
They also looked at what qualifications people had attained by the age of 33.
Test results and educational achievements all improved significantly with increasing birth weight.
The proportion of men with higher qualifications (higher than A level) was 26% in the lowest birth weight group (2500 g or less).
In babies who were born in the highest weight group (more than 4000g) it was 34%.
For women, the proportion rose from 17 to 28% for the comparative groups.
Maths scores increased with increasing birth weight at all ages.
But babies of lower birth weight born into social classes I and II outperformed normal weight babies from classes III and IV in maths tests.
Policy changes
The researchers suggest the link between maths score and social class seems to strengthen with age, whilst the association with birth weight remained similar.
They said their findings held true, even when factors such as gender, maternal age, whether the child was breast or bottle fed, number of siblings and parental education was taken into account.
Researcher Barbara Jefferis told BBC News Online: "We have been able to look at the effect of size at birth on a wide range of cognitive tests.
"In terms of policy, our findings suggest you really need to take seriously ways of addressing deprivation in childhood and improving social environments."