They were filming scenes for the Hogwarts Express in Scotland when more than 100 acres of heath land was set alight.
The heath was around the Glenfinnan viaduct, which was used in the Chamber of Secrets when Harry and Ron were flying the magical Ford Anglia.
The fire was finally put out on Sunday morning and investigators are now trying to work out how it happened.
It's too early for police to know exactly what caused the fire, but people living near the area are blaming the steam engine used by the film crew.
They said sparks from the engine started the blaze, that at one point forced 12 homes to be evacuated.
Hear story
Chris Columbus: Full interview
Swap goss on the Potter films
