South Africa's National Assembly has approved a law requiring that citizens working as security staff abroad must seek permission from the government.
South Africans were among the group arrested in Zimbabwe in 2004
It will also make South Africans seek permission to serve in foreign armies.
Defence Minister Mosiuoa Lekota said the law was to prevent mercenaries from subverting democracy across Africa but opponents say it is too stringent.
South Africa is trying to get rid of a reputation as a haven for mercenaries and coup plotters.
The authorities estimate at least 4,000 South Africans are employed in conflict areas, and several have been found to be involved in attempted coups.
"Mercenaries are the scourge of poor areas of the world, especially Africa," Mr Lekota said.
"Killers for hire, they rent out their skills to the highest bidder regardless of the political agenda."
The governing ANC's two-thirds majority in parliament ensured the bill was passed with a large majority, though several opposition parties opposed it.
Critics say the bill could destroy the jobs of those South Africans who are currently doing genuine security work in countries such as Iraq and Afghanistan.
"This could mean that thousands of South Africans currently doing legitimate work in such countries will now have to apply for authorisation and, if this is not given, will have to give up their jobs and return to South Africa where chances of employment are slim," Len le Roux of the Institute for Security Studies in Pretoria argues.
The UK High Commissioner to South Africa, Paul Boateng, was among those who appealed for changes to the bill, which is likely to affect 800 South Africans serving in the UK armed forces.
Mr Lekota defended South Africa's right to ban its citizens from serving in the British army if South Africa did not support the war in question.
"If Her Majesty's Government was engaged in or getting into a conflict that is inconsistent with our law (which is based on the demands of the Constitution), we would say we are not going to do that. And that we will regulate," he said on Tuesday.
The bill was passed by an overwhelming majority in the National Assembly, and must now be approved by the Council of Provinces, the upper house of parliament.
South Africa's role as a mercenary base was highlighted in 2004, when more than 60 South African citizens - most of them former Angolans who had fought alongside South African troops in the Angolan civil war - were arrested in Zimbabwe in connection with an alleged coup attempt in Equatorial Guinea.
All served a year in jail in Zimbabwe, and eight suspected ringleaders were subsequently charged in South Africa.
Also last year, Sir Mark Thatcher - son of former UK Prime Minister Margaret Thatcher - fell foul of South Africa's existing anti-mercenary laws in relation to the alleged coup plot and was given a suspended jail term and fined after agreeing a plea bargain to help investigators.
The alleged ringleader of the plot, Briton Simon Mann, remains in prison in Zimbabwe on a longer sentence.
In Equatorial Guinea, 14 other people were found guilty of charges linked to the alleged coup attempt, including plot leader Nick du Toit who received a 34-year jail sentence.