Tokyo says the five, visiting Japan for the first time since they were abducted to help train North Korean spies, will remain in Japan.
This is despite the two sides' failing to reach agreement on the issue during talks in Malaysia earlier this week.
North Korea has demanded Japan return the five as it originally promised.
Japan's Chief Cabinet Secretary Yasuo Fukuda said Tokyo wanted to carry on talking to Pyongyang about bringing the victims' North Korean families to Japan as well.
During the Malaysian talks, Japan also failed to persuade North Korea to give up its alleged nuclear weapons programme.
But the Stalinist state's negotiator did say the nuclear issue could be resolved with the aid of the United States.
"If the Americans will help our country and promise not to attack us we can solve the nuclear problem," said Mr Pak, as the talks in Malaysia ended.
Locked horns
During the talks Japan repeatedly said its demands on the two issues must be resolved before normalisation could go ahead.
But North Korea has insisted normalisation should come first.
Despite the lack of progress, the Japanese delegation said it would continue to negotiate with North Korea.
Japan's missing
North Korea reportedly guaranteed the safety of the kidnap victims' children who remain in the secretive state.
North Korea has suggested another round of talks at the end of November, but Japan has not yet formally accepted.
Both sides have much at stake. Impoverished North Korea badly needs the billions of dollars in Japanese aid that normalisation of ties would bring.
Chief Japanese negotiator, Katsunari Suzuki, told reporters as he landed in Tokyo: "We need to review our tactics in Japan and make contact (with North Korea) through various channels."
Nuclear fears
Mr Suzuki also said North Korea agreed to discuss security matters with Tokyo in a new forum to be set up next month.
Japan is under considerable pressure from the US to press North Korea on the nuclear issue.
The row stems from a US report earlier this month that North Korea had admitted to the programme when confronted with evidence, in contravention of an important 1994 accord.
On Wednesday five US Congressmen urged President George W Bush to abandon America's side of the agreement to provide North Korea with fuel.
But South Korean President, Kim Dae-Jung, urged the US this week not to impose sanctions on the North, warning that this would risk a war on the Korean peninsula.