Libya would be violating international and domestic law if it goes ahead with plans to deport an estimated 1m illegal immigrants, Human Rights Watch says.
Libya is a major staging posts for Africans wanting to go to Europe
The New York-based group said Libyan law prohibited deportation to countries where refugees may face persecution.
A HRW spokesman told the BBC the Libyan leader may be reacting to pressure to stop Africans trying to reach Europe.
The BBC's Rana Jawad in the capital, Tripoli, says illegal immigrants are the backbone of the country's economy.
The European Union has urged North African countries to help stop the northward flow of African migrants.
Last month, Italy and Libya agreed to set up joint coastal patrols.
The mass expulsion plan was announced just as Libyan leader Colonel Muamar Gadaffi arrived in Burkina Faso for the annual summit of West African states, which starts on Friday.
The official news agency said there would be no exceptions to the government policy of repatriating unauthorised immigrants.
"This news is very troubling," HRW's Fred Abrahams told the BBC's Network Africa programme.
"Many of the did come without papers for economic reasons, but amongst them are also legitimate refugees."
He gave Eritrea, Ethiopia, Somalia and Sudan as examples of countries that some people were fleeing.
Our correspondent says this is not the first time the North African country has threatened such a move.
Many illegal immigrants are unlikely to take the announcement seriously, she says.
Last year, the prime minister announced that all workers had two months to either secure a contract and legalise their status, or leave the country, but in the event nothing happened.
Mr Abrahams said the government should work to regulate the illegal immigrants and introduce asylum laws.