Stone-throwers forced the train to stop and set it alight
Police blamed Muslims for the attack and imposed a curfew in the area.
Appealing for calm and an end to religious violence, Prime Minister Atal Behari Vajpayee called off his trip to a Commonwealth summit in Australia.
The town has a history of Hindu-Muslim violence.
Fifteen children were among the dead.
Several carriages were gutted and some victims were burned beyond recognition, officials said.
Ayodhya row
The Hindus on the Sabarmati Express train on Wednesday had travelled to Gujarat from Ayodhya - the focus of bitter clashes with Muslims 10 years ago after Hindu militants tore down a mosque there.
Some 2,000 people were killed in nationwide riots that broke out after the demolition of Ayodhya's 16th-century Babri mosque.
The right-wing Vishwa Hindu Parishad (VHP) group has vowed to build a temple on the ruins of the mosque - and thousands of people are now reported to be gathering at the site.
After what Gujarat's Home Minister, Gobhardan Jhorapia, told the BBC was a "pre-planned" attack, Mr Vajpayee urged the VHP to step back from the campaign to build the Ayodhya temple.
"I appeal to the people to be patient, exercise restraint, and be calm," he said.
"Do not let emotions sway you. The unity and communal harmony of India will have to be protected at all costs. The government is very worried about this incident."
In addition to the children's bodies, the charred remains of 17 men and 25 women have also been recovered.
Dozens more passengers were injured and 38 were said to be in critical condition.
"I heard screams for help as I came out of the house. I saw a huge ball of fire," said Rakesh Kimani, 18, who lives nearby.
"I saw... people putting out their hands and heads through the windows trying to escape. It was a horrible sight," he said.
Journey of horror
"We have been through hell," said passenger Sanjay Kumar who arrived in Ahmedabad after the undamaged part of the train was allowed to continue.
"We don't know the fate of many passengers who were with us."
The BBC's Jill McGivering, reporting from Delhi, says the trouble appears to have started after some of the Hindus taunted a crowd of Muslims, who responded by hurling stones.
Since the train attack:
Raju Bhargav, Godhra's superintendent of police, said Hindus and Muslims had clashed during last weekend's Muslim festival of Eid al-Adha in the nearby town of Baruch.
"This could have been in retaliation to the trouble on Eid," he said.