People are not being released from camps until they have been screened
Most of Sri Lanka's displaced people could still be kept in government-run camps in one year's time, a UN official has told the BBC quoting army sources.
But the government rejected the suggestion, saying that it aimed to resettle most by the end of this year.
About 250,000 people fled the final bloody phase of the civil war between the government and Tamil Tiger rebels.
Meanwhile, a human rights group accused the government of failing to probe rights abuses during the conflict.
It was in the final weeks of the war that hundreds of thousands of civilians streamed out of rapidly-diminishing rebel-held territory.
They were ultimately housed in government-run camps in the district of Vavuniya.
The UN expressed concern about the permanent nature of the shelters being put up in these camps.
The official, Mark Cutts, said that nothing less than a new city had been created at Manik Farm, the massive complex of camps where he worked for the past month as a senior co-ordinator.
He said bulldozers were working constantly to clear jungle and that phone lines, schools, banks and even a cash machine had been built. He said this was "phenomenal" but described government plans to replace tents with more permanent structures as a "big worry".
"Senior military officials have also told us that they don't expect to see any significant returns in the next six months, On the contrary, some senior officials told us just yesterday that they expect probably not more than 20% of these people will have returned in the next year," Mr Cutts said.
But Sri Lanka's human rights minister Mahinda Samarasinghe said it was "absolutely false" to suggest that it would take so long. He said it was not the military but the government who took such decisions and that it aimed to resettle most people by the end of this year.
In the past week, the government says about 2,000 displaced people have been resettled in their villages in the north-west. These people fled their homes about two-and-a-half years ago.
It says that the refugees living in the camps are being strictly vetted to ensure they have no links with the rebels. Only after that process can their return home be considered.
But the UN said that even those who have been through the screening process have not yet been released from the camps and that no one apart from those under 10 or over 60 years of age were being allowed out.
"We need to look into this issue of how long are they going to be kept in these places, will they be given proper freedom of movement - and that is going to be a big concern if these camps are going to be there for longer than the three to six months we initially assumed," Mr Cutts said.
He said that humanitarian access to the camps had improved in recent days, but this was not unconditional. While understanding the government's security concerns, he said the UN had to ensure international standards were met when dealing with the displaced.
Mr Samarasinghe confirmed that those being screened must wait for the process to be completed but said they needed somewhere to stay in the meantime.
About 300,000 are being kept in government-run camps
In a separate development, human rights group Amnesty International accused the Sri Lankan government of never seriously investigating the human rights abuses allegedly committed during the 26-year civil war.
It has called on the government to take advantage of the end of the conflict to seriously investigate all these allegations.
It says the issue is even more pressing in the wake of the controversies that arose during the final weeks of the conflict, when international human rights groups accused both the Tamil Tiger rebels and the government of committing war crimes.
The government was accused of subjecting areas of rebel-held territory to indiscriminate shelling, while the rebels were accused of using civilians as human shields.
Sri Lanka has previously dismissed calls for an independent inquiry into claims of human rights abuses by the military, saying its own courts will investigate.