The ambush is the second in a week in the occupied territories, in an apparent change of Israeli tactics which had previously depended on air strikes to punish armed Palestinian attacks.
The army said the dead men were "terrorists" who had opened fire overnight by Alfei Menashe settlement near the Palestinian-controlled town of Qalqilya - an account strongly disputed by the Palestinian side.
The ambush is the second in a week in the occupied territories, in an apparent change of Israeli tactics which had previously depended on air strikes to punish armed Palestinian attacks.
The latest killings - and the weekend border violence in Lebanon - come at the beginning of the Muslim holy month of Ramadan when analysts say tensions might be expected to rise.
Palestinian leader Yasser Arafat's Fatah faction joined the Palestinians' militant Islamist opposition in calling for an escalation of the uprising during Ramadan.
'Cold blood'
Palestinian radio said the five men - named as Qusay Zahran, Mahmud al-Adl, Muhammad Abu Adwan, and teenagers Mahdi Jabir and Ziyad Silmi - were Fatah members who had headed to the scene after hearing gunshots.
But the Qalqilya Fatah official, Ahmad Haze'a, called it a cold blooded killing and said the soldiers had prevented medical assistance reaching the men as they bled to death.
Futile diplomacy
The latest bloodshed demonstrates the failure of a recent flurry of crisis meetings and telephone calls among politicians and security officials to put diplomacy back on track.
The Palestinian security chief in Gaza, Abdul Raziq Majeidah, met Israel's southern region army commander Yom Tov Samia on Sunday, but it was described as a "waste of time" by the Palestinian side.
Meanwhile in Cairo, the head of Israel's Shin Beth domestic security service, Avi Dichter, held talks with Muhammad Dahlan, the Palestinian preventive security chief, according to media reports.
The talks followed Mr Arafat's brief meeting with Israel's tourism minister Amnon Lipkin-Shakah over the weekend.
The two sides have already agreed terms designed to end the bloodshed which has claimed about 290 lives, most of them Palestinian, but they have done little to ease the violence.
The Israeli Foreign Minister, Shlomo Ben Ami, has postponed a trip to Moscow because the main opposition party, the right-wing Likud, is planning to force a vote in parliament on Tuesday on holding new elections.