Farming has damaged the elephant's natural habitat
The elephants were to be moved from the southern province of Binh Thuan, where deforestation has ruined their natural habitat, to a national park in Dak Lak province.
The death of two elephants means there are only an estimated six animals left in this particular herd.
A team of Malaysian and Vietnamese elephant experts spotted the two elephants on Monday and shot them with tranquilliser darts.
The smaller elephant escaped, but died later when, apparently exhausted by the drugs, it fell down a steep hillside, said Nguyen Van Dung, deputy chairman of the People's Committee of Binh Thuan province.
Media attention
The larger elephant was captured and chained, but was found dead on Wednesday, said Vietnam Television.
A Malaysian expert told VTV the elephant may have been alarmed by photographers and reporters who approached it, and possibly fell over a tree stump.
Conservation group Fauna and Flora International (FFI), which has given funding and technical advice with assistance from the Dutch Government, said security should have been tighter.
"It is absolutely imperative that the local police prevent onlookers putting either themselves or the animals in danger," said FFI's country director Frank Momberg.
The relocation has been put on hold pending an investigation into the animals' deaths, said a forestry department official.
The herd has attracted a lot of attention in the media because it has trampled to death a dozen villagers in the last two years in the search for food. Their forest area has been shrinking under the spread of maize plantation.
Dak Lak province, where the small herd of elephants was heading, is already home to Vietnam's largest herd of Asian elephants, numbering less than 40.
The last time Vietnam tried to move a herd of elephants - in Dong Nai province in 1993 - a Singapore expert was trampled to death and all 12 elephants died because they were given overdoses of tranquillisers or jumped to their deaths from moving trucks.