Motorists who enter the centre of Tenby during the busy summer season face fines of £60 under a controversial county council scheme.
Cars will be banned from the narrow streets from 11am to 5pm
The walled area of the seaside town is being made a no go area for most vehicles during July and August in an attempt to end crippling traffic jams.
Tenby sees a huge influx of families drawn to its miles of beaches and Pembrokeshire Council is extending a pilot scheme it ran last year in a bid to make the streets safe and congestion free.
But a residents' group says the authority has gone too far and at a meeting on Thursday night voted unanimously to press for a public inquiry unless the plans are modified.
We are trying to be non-confrontational and we are trying to work with people
TWTRA chairman Richard Walker
Number plate recognition cameras will be used to police the initiative that will run from 5 July to 31 August.
Vehicles will not be allowed to enter the walled centre between 11am and 5pm without prior approval or the driver will be fined.
Last year the authority set the penalty at £25 but it has been increased to £60 this year.
Around 50 people attended Thursday's meeting of the Tenby Walled Town Residents Association (TWTRA).
They said family doctors on routine calls, delivery drivers, couriers and taxis should all be given dispensation.
The council was accused of setting the penalty too high and of not giving those who lived within the walls enough leeway.
Chairman Richard Walker said: "These things are solvable and I just wish the county council would be open for discussion.
The scheme in Tenby will run from 5 July to 31 August
"We are trying to be non-confrontational and we are trying to work with people."
Vice-chairman Alistair Mackay said: "We don't feel they should open the flood gates so there's access to everyone but they must reasonable.
"We are not going to get a public inquiry before pedestrianisation but we do not believe they have followed their own guidelines."
Letters from Pembrokeshire MP Nick Ainger to the council asking for the objectors concerns to be discussed were read out.
Members of TWTRA have engaged the services of their own traffic consultant and say if necessary they are ready for a legal fight.
Pembrokeshire Council carried out of a survey following last year's pilot and two-thirds of those who responded backed the scheme.
Pembrokeshire Council says its questionnaire was sent to all households on the electoral register in the Tenby, Caldey, Penally and New Hedges electoral divisions.
Over 1,500 replies were received - a response rate of 44%.
Of the replies, 88% said there was a need to improve the traffic situation in Tenby.
The council's head of transport Huw Roberts said: "The majority of those who expressed a view wanted to see the pedestrianisation start earlier in the day, finish later in the day, operate in the evening and extend to other months of the year.
"The Council has listened to those views and also to the significant minority who said they did not want to see the experiment repeated.
"Tenby is an important tourist destination in the county.
"The initiative aims to improve the enjoyment of the town by residents and visitors alike and to encourage more people to return to Tenby in the future."
There will be a consultation meeting about the scheme on June 19 and the council's cabinet will decided whether to ratify it on 23 June.