Japan's navy went on alert on Wednesday after an unidentified submarine was spotted inside the country's waters.
Japan's defence chief called for a peaceful resolution
Surveillance aircraft detected the submarine near the southern island of Okinawa, and it left soon afterwards.
Officials said the government was trying to establish the vessel's country of origin, although Japanese media said it was possibly Chinese.
The last time Japan ordered such a high alert was in 1999, when suspected North Korean ships entered its waters.
Japan's Prime Minister Junichiro Koizumi called the incident "regrettable".
"We are still in the process of confirming the nationality. Once we confirm it, we will disclose it," he said.
Chief Cabinet Secretary Hiroyuki Hosoda said the submarine was spotted by a reconnaissance plane near the Sakishima island chain off Okinawa.
He said the submarine had now left Japan's waters, but that it was being followed by a reconnaissance plane and a destroyer in the hope it would surface and show its flag.
Japan's defence chief Yoshinori Ono issued the maritime alert, a very rare event in Japan, which is bound a post-World War II constitution which restricts its forces to self-defence.
"We want to resolve this issue peacefully," he said.
Japanese officials refused to speculate on the submarine's nationality. But Japan's Kyodo news agency quoted defence officials as saying it was a Chinese nuclear-powered sub.
The Sakishima islands lie about 120km (75 miles) south of the disputed Senkaku islands, which are known as the Diaoyu in Chinese.
The incident comes at a time of heightened tensions between the two countries, which are likely to become increasingly competitive in their hunt for natural resources to power their economies.
The two sides held inconclusive talks last month on gas exploration projects in the East China Sea.
China has also reportedly been angered by a Japanese defence ministry paper which speculated on reasons China might attack. It cited disputes over natural resources and territory, as well as a wider conflict involving Taiwan.
The other possible source of the submarine is North Korea.
In December 2001, a suspected North Korean spy ship sank off Japan after a shoot-out with Japan's coast guard.