Around 16 Gypsy families in Somerset have had their application for planning permission turned down.
The Gypsies have been told to stop building on the site
They had started to build homes on a five-acre site they own in Oxen Lane, North Curry, without obtaining the necessary permissions.
At a meeting on Thursday, Taunton Deane Borough Council voted to deny planning permission. The group is to appeal.
A vote was also passed to seek an injunction to have the GYpsies evicted from the site.
The council first started to take action against the Gypsies after they started installing septic tanks and bringing in stone to build makeshift roads.
The group did not have planning permission for any new developments.
The council issued the families with a notice on 29 October that they must stop building work and restore the land to its former agricultural use.
Some residents have expressed anger at the site. Hilary Harris said: "They have devastated our lives and we want them to go."
The Romany Gypsy Council said there is a shortage of private plots for gypsies to buy and live on, and that the site would not be there if there was not a need for it.