The court in the Hague rejected Israel's claim that it didn't have the right to decide if the barrier was legal.
The court also said that Israel should give money to people who owned land that was taken to build the barrier.
The Palestinians are expected to try to get the UN General Assembly to demand that Israel take the barrier down.
Palestinian Prime Minister Ahmed Qurei said: "This racist wall is illegal to the root."
Israel say the barrier has already started doing its job by stopping suicide bomb attacks against Israelis.