Mr Mandela was speaking in the Tanzanian town of Arusha where he launched a new bid to revive the peace process.
The former president squarely blamed Burundi's politicians for failing to show the talent and vision needed to end the brutal civil war that has cost some 200,000 lives in the last six years.
"Please join the modern world. Why do you allow yourselves to be regarded as leaders without talent, leaders without vision?" he asked.
He accused the leaders of Burundi of showing no urgency in their efforts to end the fighting, and told them that they would have to compromise to end what he said was a senseless slaughter.
"When people in the West hear these things they say 'Africans are still barbarians - no human being could do what they are doing'.
"The fact that women, children and the aged are being slaughtered every day is an indictment against all of you," he said.
The listening politicians responded to the speech with a standing ovation.
The Burundi Government described the remarks as fair and understandable.
Minister for Peace, Ambroise Niyosaba, told the BBC that Mr Mandela's comments were a clear indication that he was committed to helping the country find a lasting solution to its civil war.
"He called us to order," the former president of Burundi, Silvestre Ntibantunganya, said. "It is our duty to respond."
"There are things he said which won't please everyone, but they were things which needed to be said," said Leonce Ndarubagiye of a faction of the CNDD rebel group.
International effort
Mr Mandela also announced that he had invited presidents Bill Clinton of the United States, Jacques Chirac of France and other world leaders for peace talks on Burundi in Arusha next month.
Similar invitations to attend the opening sessions of the talks had also been sent to Saudi Arabia, Nigeria, Uganda, Tanzania and the Organisation of African Unity.
Mr Mandela was named as the new mediator in peace talks in December, replacing former Tanzanian President Julius Nyerere, who died in October.
Previous efforts under Mr Nyerere were considered to have failed because rebel Hutu groups were excluded from the negotiations.
Mr Mandela conceded that the peace process had to involve all parties to the conflict.
"The process must be all-inclusive, otherwise there can be no guarantee that the decision of the people here, even if it is unanimous, will be respected by the armed groups on the ground."
Exodus continues
The number of people fleeing into Tanzania to escape the war in Burundi shows no sign of lessening.
A BBC correspondent in the Tanzanian town of Kibondo, says that it is simple fear that is causing the exodus.
The UN High Commissioner for Refugees, Sadako Ogata, will be visiting refugee camps near the border on Monday.
She has appealed to the international community to devise concrete measures to ease the plight of millions who have been forced from their homes around Africa.
Related to this story:
New peace talks for Burundi
(06 Dec 99 | Africa)
Burundi rebels reject Mandela talks
(27 Dec 99 | Africa)
Burundi army blamed for massacre
(07 Jan 00 | Africa)
Burundi's deadly deadlock
(03 Nov 99 | Africa)
Mandela to head Burundi peace drive
(01 Dec 99 | Africa)
UN alarmed over Burundi camps
(13 Nov 99 | Africa)
US urges Burundi peace talks
(22 Oct 99 | Africa)
UN suspends Burundi operations
(14 Oct 99 | Africa)
Burundi tackles cholera in camps
(21 Dec 99 | Africa)
Eyewitness: Inside Burundi's camps
(13 Oct 99 | Africa)