Mr Hekmatyar's forces have previously been allied with the Taliban
Afghan President Hamid Karzai has met a delegation from the country's second biggest militant group, officials say.
The Kabul talks are the first confirmed direct contact between Mr Karzai and envoys of former premier Gulbuddin Hekmatyar's Hezb-e-Islami faction.
Mr Karzai has yet to respond to a tentative peace plan from the group at talks two days ago, his spokesman said.
Talks with insurgents are seen as vital to securing peace although any deal is a long way off, BBC correspondents say.
Hezb-e-Islami fighters are based mainly in eastern Afghanistan and share many aims with the Taliban - the biggest militant group in the country. There have been recent tensions however, with the two groups clashing in the north.
Observers say the talks in Kabul may only be preliminary but they come at a fluid time in Afghan politics, with a peace jirga, or tribal gathering, due to be held some time next month and a surge in US-led troop numbers under way.
On Friday the former UN envoy to Afghanistan, Kai Eide, confirmed he had been holding secret contacts with top Taliban leaders for the past year.
The BBC's Quentin Somerville in Kabul says there is a growing recognition, both within Afghanistan and from its foreign partners, that insurgents have to be part of any peace settlement and that military operations alone will not be enough to bring peace to the country.
The five-member Hezb-e-Islami delegation in Kabul is headed by a former Afghan prime minister, Qutbuddin Helal, who is deputy to Mr Hekmatyar - himself another former prime minister.
The two sides have been in contact before but this is the highest profile meeting yet.
"I can confirm that a delegation of Hezb-i-Islami... is in Kabul with a plan and has met with the president," Reuters news agency quoted a spokesman for Mr Karzai as saying.
Among the group's demands are a pull-out of foreign forces from Afghanistan by this summer, a year ahead of a date indicated by US President Barack Obama for any withdrawal to begin.
They also want fresh elections within a year and a new constitution.
"The main condition is the empowerment of President Karzai to engage in talks and make decisions," a spokesman for Hekmatyar, Wali Ullah, said.
"The aggressive occupying forces should also announce a schedule for leaving Afghanistan."
Clashes with Taliban
Mr Hekmatayar is a highly controversial figure: his group has battled Nato and Afghan forces in Afghanistan's east and north for years, while the Taliban have led the insurgency in the south.
Hezb-e-Islami fighters surrendered their weapons in Baghlan
Earlier this month, officials said that at least 60 militants were killed in fighting between the Taliban and Hezb-e-Islami in Baghlan province in northern Afghanistan.
Reports said they had clashed over control of local villages and taxes.
The two groups have previously been allied in their opposition to Afghanistan's central government and the presence of foreign forces.
Along with the Taliban, Hezb-e-Islami has been blamed for much of the insurgent violence in Afghanistan.
Mr Hekmatyar was one of the main recipients of US military aid during the Soviet invasion of Afghanistan in the 1980s, but was later vilified for his part in the fighting among mujahideen factions which killed more than 25,000 civilians in the early 1990s.
He was designated a terrorist by the United States in 2003 for supporting al-Qaeda.