Israelis and Palestinians began shoving each other in the courtroom, until police put the two groups in separate rooms.
Defence lawyers acting for the imprisoned leader - head of Yasser Arafat's Fatah movement in the West Bank - now say they are going to resign, accusing security men of beating him and throwing him out of court as the hearing was about to begin.
Israeli prosecutors have charged Mr Barghouti - the most senior Palestinian to be tried by Israel - with heading a militia accused of murdering 26 Israelis. If found guilty, he faces life in prison.
Mr Barghouti's lawyers have also refused to recognise the court, arguing it does not have the legal right to try their client.
Murder accusation
BBC correspondent Barbara Plett, who was at the court in Tel Aviv, says the trial is also about different views of the Palestinian uprising.
By holding a criminal trial for an intifada leader, Israel wants to prove it is committed to fighting terrorism.
Mr Barghouti, however, wants to turn it into a trial of the occupation itself.
In a statement, he called on the Palestinians to continue their uprising and to defy Israeli military curfews.
"Though I am the one in chains, it is not I who is on trial in Israel," Mr Barghouti said.
"What is on trial today is the conscience of all freedom-loving people around the world."
'We will be victorious'
Defence lawyer Khader Shkeirat accused Israeli police of beating and shoving him when he tried to get to Mr Barghouti in the crowded courtroom.
"I am so worried about Marwan's life. He could be assassinated by the criminals, by the settlers, by the fanatic people inside the court," Mr Shkeirat said.
As the courtroom descended into near-chaos, a woman whose daughter was killed in an attack Mr Barghouti allegedly helped orchestrate held up a picture of her slain child and shouted: "He is a vampire taking away the blood of Jewish children."
"They should castrate you," one man yelled.
But Mr Barghouti himself was in defiant mood and before being led away from the courtroom he shouted: "We will be victorious over the occupation."
Since violence erupted in September 2000, Marwan Barghouti has become one of the most vocal supporters of the uprising, giving fiery speeches and becoming a charismatic leader.
He was seized during an Israeli army invasion of the West Bank town of Ramallah in April.
In both Ramallah and Gaza City on Thursday, hundreds of Palestinians demonstrated on Thursday in support of Mr Barghouti.
His wife Fadwa, who was prevented from attending the trial after Israel refused to grant her a travel permit, told a Ramallah rally the proceedings against him were "an opportunity for the Palestinian people to put the occupation on trial".