UN Secretary-General Kofi Annan "is minded to disband" the fact-finding team after more than a week of wrangling and delays, a top official in New York said.
The latest word from the UN came after the Israeli Government again refused to co-operate with the team.
Senior Palestinians expressed dismay at the possible fate of the initiative, urging Mr Annan not to give in to "Israeli pressure".
In other developments, the Israeli military announced it had completed its withdrawal from the West Bank town of Hebron following a two-day incursion.
According to an army statement, 150 Palestinians were detained including more than 50 on Israel's wanted list.
In the Gaza Strip, Palestinian medical workers said a young child was killed by Israeli tank fire at Rafah late on Tuesday, but Israeli military sources dispute the circumstances of the incident.
Earlier, more Palestinians were released from the Bethlehem church besieged by Israeli forces for a month - but they are not thought to include any of the armed militants at the heart of the stand-off.
Objections
Israel has raised a series of objections over the mandate of the UN team, which is now in Geneva.
The Israeli Government said after a meeting on Tuesday that concerns still needed to be addressed.
"Israel has presented crucial conditions so that the mission can honestly seek the facts, but as long as these conditions are not met it is not possible for the mission to start its work," an official statement said.
Its conditions include:
The UN had done everything to meet the Israeli concerns, Mr Annan said earlier.
The Security Council was told by a senior official, Kieran Prendergast, that without the cooperation of the Israeli Government, it would be impossible for the team to carry out its aims of establishing the circumstances leading to the destruction at the Jenin camp.
According to the BBC's Greg Barrow at the UN in New York, although Mr Annan appears to be leaning in favour of disbanding the UN team, diplomats there say it is still unclear whether a firm decision has yet been made on this.
Mr Annan is likely to take into account the views of Security Council members before pulling the team back from Geneva, our correspondent says.
Senior Palestinian negotiator Saeb Erekat urged Mr Annan not to back down in the face of the difficulties thrown up by Israel.
"We call on him not to give in to Israeli pressure... If he doesn't send the team it will be extremely harmful for the United Nations and for international law", Mr Erekat was quoted as saying.
Month-long standoff
On Tuesday afternoon, 26 Palestinians left the Church of the Nativity in Bethlehem. Emerging one by one, they were escorted out of the church by a monk and taken into Israeli custody for questioning.
Israeli army spokesman Colonel Olivier Rafowicz said half the freed group were members of the Palestinian security forces.
None are believed to be from a group of about 30 armed militants who fled to the church after Israel stormed into Bethlehem at the start of April.
Civilians, police, monks and nuns are among the 200 or so people still inside the church, but the militants are at the heart of the continuing stand-off.
Israel has offered them trial or exile, but Palestinian negotiators reject this and propose they be sent to Gaza. There has also been discussion of using American and British guards to supervise the men.