The powerful New Zealand centre has decided to return to Australia in the hope of regaining his Queensland State of Origin place.
Carroll, who joined the club from the Brisbane Broncos after the 2000 World Cup, was the Rhinos' top try-scorer last season with 23.
The 26-year-old said: "I have really enjoyed my time at Leeds and I would love to return here at some point in my career.
"The people of Leeds have really welcomed me to their city and I shall always have fond memories.
"The main thing I have missed not playing in Australia has been the State of Origin and that has been the main factor in what has been a very difficult decision.
Interested
"That is for next season though and my focus now is to try and help the Rhinos reach the Grand Final in October."
Carroll, who has struggled with injuries this season, had a clause in his three-year contract allowing a possible release at the end of the 2002 season.
While with the Broncos he made eight Queensland appearances and several NRL clubs will now be interested in his availability.
Rhinos chief executive Gary Hetherington said: "Tonie has been an excellent player for us during his two years here and we respect his decision to return home.
"I know it has been a difficult decision for him and hopefully this will not be the last we see of Tonie Carroll at Headingley."