Until the military moved in illegal diggers were seeking their fortune
Lobby group Human Rights Watch has accused Zimbabwe's army of using forced labour, including children, to mine diamonds in the east of the country.
Local villagers who do not co-operate with the military are beaten and tortured, the US-based group says.
Their report also details an alleged massacre of diamond diggers last year, after the disputed elections.
It urges the unity government to take control of the mines and use the revenue to help rebuild the country.
"Zimbabwe's new government should get the army out of the fields, put a stop to the abuse," Human Rights Watch's Africa director Georgette Gagnon said.
"The police and army have turned this peaceful area into a nightmare of lawlessness and horrific violence," she said.
'Buying off the military'
The report is based on interviews done in February in Marange district.
Its researchers say that as far as they are aware, the situation has not changed since the former opposition joined the government four months ago.
Human Rights Watch claims control of the mines is part of a systematic attempt by President Robert Mugabe's Zanu-PF party to buy support from the military.
The diamond fields in Marange were seized just one month after the power-sharing deal was first agreed in September 2008.
On the face of it, the military takeover was an attempt to seize control from unlicensed miners, the lobby group says.
But in reality it was a systematic attempt to enable key army units, whose support President Mugabe needed following June's elections, to have access to riches, Human Rights Watch says.
"Documents that we reviewed that we got from the military and the police clearly indicate that this was a clearly designed system to benefit the army," researcher Dewa Mavhinga said.
Witnesses say it involved a brutal military operation that saw some 200 people killed in three weeks.
It says army brigades are still in control forcing hundreds of children and adults endure forced labour for mining syndicates.
While the new Prime Minister Morgan Tsvangirai is touring the West lobbying for aid, "millions of dollars in potential government revenue are being siphoned off through illegal diamond mining, smuggling of gemstones
and corruption", the rights organisation says.
If the diamond industry was legally regulated, Human Rights Watch estimates it could amount to $200m a month for the country.
It is calling for diamond exports from Zimbabwe to be banned and for the country to be suspended from the Kimberly Process - the certification scheme for diamonds - until the demilitarisation of the mines is achieved.
On Wednesday, Global Witness reported that the Kimberly process was failing - partly because of the situation in Zimbabwe.