Most victims of the violence have been Muslim
The prime minister, in his first visit to Gujarat since violence broke out in late February, told riot victims that the failure of the local administration in preventing the clashes would be investigated.
In fresh violence on the eve of the visit, eight people were killed - and there are fears that the violence may spread to neighbouring states.
The human rights group Amnesty International has called on the authorities to investigate the violence.
'Madness'
Mr Vajpayee is reviewing efforts to control the violence and look at measures to rehabilitate people forced from their homes.
In comments which appeared to back criticism of the state authorities, Mr Vajpayee said he would speak to political leaders about allegations that they had failed to do their job.
"Government officials, political leaders, need to respond to the task. The constitution guarantees equal rights for all," he said.
The state government is controlled by the BJP, and the Chief Minister, Narendra Modi, has come in for particular attack over the way the authorities reacted to the violence.
At Godhra, the prime minister climbed inside the burnt-out coach of the Sabarmati Express, which had been torched by the mob.
Reports say at least 25 people were arrested as a "preventive measure" before Mr Vajpayee's arrival.
Heckling
At the Shah Alam camp in Gujarat's commercial capital, Ahmedabad, Mr Vajpayee said that the Godhra attack was "condemnable" but what followed was "madness".
"The answer to madness is not madness," he said in an emotional speech.
But angry victims heckled the prime minister.
Many said the prime minister's visit had come too late.
"Why was the prime minister not here a month ago? Was he waiting for more deaths?" the Indian Express quoted Mumtaz Rermatula, a Muslim woman in a relief camp, as saying.
Tens of thousands of people, mostly Muslims, are sheltering in more than 100 camps in the state.
Relief workers allege that conditions in the camps were hastily improved ahead of the prime minister's visit.
On the eve of the visit, the bodies of five Muslims were found - three men and two women. They were killed when their homes were set on fire in the village of Abasana, near Ahmedabad.
Police said another three people died in other incidents later on Wednesday - two of them shot as mobs rioted on the streets.