The rebels are fighting for an independent Islamic state
An Indonesian military spokesman said Abdullah Syafei of the Free Aceh Movement (GAM), had died during a fierce gun battle in the north of Sumatra island.
A senior GAM spokesman confirmed to the BBC that Mr Syafii, 54, and at least six others including his wife were shot dead on Tuesday night in a raid on their jungle base.
The spokesman said a new commander had already taken over on a temporary basis until a full meeting of the leadership is held.
Our Jakarta correspondent, Richard Galpin, says Mr Syafei's death is a severe blow to the rebels. He has been their commander for the past 12 years.
Lieutenant-Colonel Supartodi, said the rebel leader was shot in the head and chest during a raid on his jungle base in Pidie district.
His body had been taken to Sigli hospital, where his family could collect it for burial, he said.
Rebels have called on the authorities to hand over the bodies so they can be buried properly in their home village.
Thousands killed
Thousands of people have died since the Aceh rebels launched their revolt in 1975, after years of military abuses in the region, about 1,800 kilometres (1,100 miles) north-west of Jakarta.
At least 100 people have already been killed this year.
Over the last year the government has taken a dual approach to the problem - the military launched a major military operation after it became clear that peace talks were making little progress; and Jakarta has granted Aceh greater self-rule and a larger share of oil and gas revenues.
It also allowed the province to implement Islamic law but ruled out independence.
But a recent report by New York-based Human Rights Watch said Jakarta was failing to address human rights violations in Aceh.
"The 2001 death toll had topped 1,300 by September, and while most of the deaths were civilians caused in the course of military operations, the rebel Free Aceh Movement was also responsible for serious abuses," it said.