Pyongyang has reportedly told the US to make a deal soon or North Korea would spend the time developing nuclear arms.
The visitors were told all the spent rods had been reprocessed
The warning was delivered to an unofficial team from the United States visiting the Yongbyon nuclear reactor.
"Time is not on the US side," a member of team, Charles "Jack" Pritchard, said he was told.
But the former US state department official said there was no way to check the North's claim to have reprocessed nuclear fuel rods for weapons use.
Mr Pritchard's comments are the first lengthy public statements by any of the five men who were taken to visit North Korea's secret nuclear complex earlier this month.
While the visit was deemed "unofficial", team members have close links to the US Government and the move was seen as significant.
The US believes North Korea is likely to have one or two nuclear weapons and may be trying to develop more.
Washington, in concert with a number of Asian powers, wants to stop the development of what Pyongyang calls a "deterrent".
They are also trying to prevent the possibility of weapons or technology being exported.
Both sides say they want a solution but six-party talks launched last year have not resumed.
The US experts spent several days in North Korea and visited Yongbyon, the first foreigners to do so since UN inspectors were thrown out and the plant reactivated over a year ago.
Mr Pritchard said he was told by the Vice-Foreign Minister, Kim Kye-gwan: "Time is not on the US side. Lapses of time will result in quantitative and qualitative increases in our nuclear deterrent."
Charles "Jack" Pritchard held official talks with North Korea in 2002
Mr Pritchard, addressing an audience at the Brookings Institution in Washington, said the team was shown an empty holding pond where 8,000 spent nuclear fuel rods had once been kept.
He said he was told all the rods had been reprocessed, allowing scientists to make plutonium bombs, but said he had no way to verify the claims.
Other members of the team, such as the former head of the US nuclear laboratory at Los Alamos, might be able to make better technical assessments, he said.
Mr Pritchard, part of an official state department visit to Pyongyang in October 2002, said he also still believed US intelligence that North Korea had a second nuclear programme involving enriched uranium, though his hosts told him that was not true.