But over the last 20 years many people have set up new homes on their land.
This, together with logging and road building, means their forest home is gradually being destroyed.
But now an international organisation called Survival is helping them fight the case in the Brazilian courts.
It's hoped this will stop other people settling on their land and secure their future.
Go to the link on the right to find out more on Survival.