The Castle and Cathedral are both used to represent Hogwarts in the first two films, but the film crew have decided that they don't need to go there this year.
They already have loads of footage from previous visits, so may use special effects to add in the actors for the third film.
Paul Mingard, a man in charge of getting films made in the North East, said the film crew spent £500,000 alone on hotels in the area when making Chamber of Secrets.
Some stuff will be filmed outside
Filming on the Prisoner of Azkaban starts on Monday at Leavesden Studios in Watford, with the outdoor shots being filmed later in Glencoe in Scotland.
Hear story
Chris Columbus: Full interview
Swap goss on the Potter films
