Griffiths is reported as saying "I said to JK Rowling: "Couldn't the Dursleys turn up to an open day at Harry's school or something,' but she said, 'I don't think so'."
The book of The Goblet of Fire is so long that there as no way that the filmmakers could fit everything into the movie.
It now looks as if the action will start at The Burrow or the Quidditch World Cup.
Although he says he won't be in this film, Richard Griffith says he's hoping to be back for Order of the Phoenix.