He also revealed that he thinks the special effects, such as Dobby and Quidditch, are much better this time.
This is partly because they filmed them first and had nine months to get them right.
With Philosopher's Stone they shot them last.
He agreed with Rupert and Emma, who we spoke to earlier this week, saying that the young actors were all more comfortable this time round.
Time for his children
And he said it wasn't a hard decision to turn down the third movie.
He says he hasn't had supper with his children on a weekday for over two years.
"I have four children and I have to give them some time now", he said.