Abdul Ghani Lone, of Kashmir's main separatist alliance, the All Party Hurriyat Conference, was shot while speaking at a rally in the city of Srinagar.
The killing came just before Indian Prime Minister Atal Behari Vajpayee arrived in the region to visit survivors of an attack by militants last week that left more than 30 people dead.
There are growing fears that India will start a war against Pakistan who it blames for backing the militants in Kashmir.
British Foreign Secretary Jack Straw is to visit the region next week in a bid to ease tensions between the two nuclear rivals.
Masked men
Mr Lone and a bodyguard were shot dead while attending a ceremony commemorating the assassination of a Kashmiri leader, witnesses said.
Kashmir conflict:
Three masked men are said to have approached him and opened fire.
The gunmen have not yet been identified and no one has admitted carrying out the attack.
Mr Lone was taken to a hospital in Srinagar, but was found to be dead on arrival, reports say.
The BBC's Altaf Hussain in Srinagar says the killing appears to have derailed an expected peace initiative by Mr Vajpayee.
Mr Lone was seen as a moderate who wanted talks with the Indian Government over his demands for a separate Kashmir state.
Indian Kashmir's Chief Minister, Farooq Abdullah said the attack was the work of the government in Islamabad.
"Whoever wants a peaceful solution in Kashmir is killed by Pakistan," he said, the AFP news agency reports.
But the Pakistan Government was quick to condemn the attack, describing his murder as a setback to the peace process.
Three militant groups in the area have blamed the attack on the Indian secret services.
Heavy shelling
Tensions between India and Pakistan have soared since more than 30 people were killed in an attack on an army camp near Jammu last week, which India blamed on Pakistan-based militants.
Heavy shelling between the two sides over their border in Kashmir continued for a fifth straight day on Tuesday.
At least three civilians were killed and seven injured in the latest exchanges, an Indian army spokesman said.
Pakistani police said a 10-year-old had died in the firing.
Shares on the stock exchange in Bombay have fallen sharply, in expectation of hostilities breaking out.
Karachi had its biggest daily fall in nearly four years on Monday.
War of words
Pakistan has urged the international community to "convince India to see reason", and appealed for talks with Delhi to defuse the situation.
Several countries, especially the United States, are worried about the conflict escalating out of control, and have urged both sides to exercise restraint.
The British Government has launched a diplomatic initiative sending Foreign Secretary Jack Straw to the region next week.
"The international community is watching events with mounting concern. This is a crisis the world cannot ignore", Mr Straw said in London.
The US is also sending Deputy Secretary of State Richard Armitage to India and Pakistan, although no date has been set.