This follows joint efforts with the United States to try to strengthen security in Yemen.
A decree by President Ali Abdullah Saleh did not say when the new agency would begin working or how it would be formed.
According to one report, the agency - working alongside the intelligence services - would be directly responsible to the president.
Security in Yemen become a major US concern following an attack on an American warship in the port of Aden which killed 17 US servicemen in October 2000.
The US also believes members of Osama Bin Laden's al-Qaeda network are hiding in Yemen.
The Yemeni Government recently said it was holding 85 suspects, all accused of having links to al-Qaeda and other militant groups.
"Foreign enemies"
According to the Associated Press, which obtained a copy of the decree, the new agency's key roles will include gathering and analysing intelligence data about foreign threats to national security.
It must also pre-empt and combat acts of terror, and prevent militants "and other foreign enemies" from slipping into the country.
Building closer ties with and exchanging information between other foreign security organisations will be another function of the new agency, the decree said.
There was no immediate comment from government officials.
In recent weeks a team of US Special Forces soldiers has been in Yemen to train its fledgling counter-terrorist unit.
A US team is also reported to have begun installing a surveillance system at sea and air ports.