Simon's neighbour Mrs Olive Irons says: "It is in the front garden in full view of everyone and it does make noise."
The local council says Simon should have asked permission to build such a big ramp in his garden.
And now, they've told him he has to take it down.
Simon is not happy about the council's decision, he told the BBC: "All the prize money I made last summer... built the ramp.
"All my main rivals in America have extensive training facilities on their doorstep and I was missing out."
Simon is planning to appeal the council's decision.