By Jim Reed
Newsbeat reporter in Amsterdam
Authorities want to cut crime levels in the city
The Mayor of Amsterdam has told Newsbeat he will push through plans to close down half the cannabis cafes in the centre of the city and some sections of the red light district.
Authorities believe the unrestricted growth of cannabis coffee shops and legal brothels has encouraged crime like money laundering and people trafficking.
But critics of the clean up say it will just drive the sex and drugs trade further underground and make the area less safe for the girls working there.
Mayor Job Cohen told Newsbeat: "The measures we are taking now, especially in the red light district, are necessary because we think there is too much criminality.
"We are not going to close the red light district and there will still be soft drugs but less than there is now."
Many of the million British tourists who visit Amsterdam each year are attracted by the city's liberal attitude to sex and soft drugs.
Prostitution has been legal for the last eight years although it was unofficially tolerated for much longer than that.
There are 228 coffee shops across the city licensed to sell up to five grams of cannabis for personal use.
National politicians want to close 43 by the end of 2011 because they are within 250 metres of schools.
A separate plan backed by the local council and mayor's office could see the closure of another 38 cafes in the city centre to stop the "overrepresentation of crime sensitive sectors" in the area.
The number of open prostitutes' windows will be cut from 482 (in 2007) to 243 with the trade restricted to two main streets running through the old medieval red light district.
Local council leaders say the district needs to diversify and attract higher paying visitors more interested in the city's art galleries and museums.
"Those British tourists coming here for a weekend and using a lot of alcohol and a lot of soft and hard drugs are not the best of our tourists," said Mr Cohen.
"Of course they are welcome but not if they abuse alcohol and drugs."
Prostitutes' union, De Rode Draad (Red Thread), which represents 20,000 sex workers, is fiercely opposed to the plan.
Spokeswoman Metje Blaak said: "It's a very bad plan that just pushes the trade underground where the criminals can get at them."
A prostitute sits in her window in Amsterdam's red light district
"He's making the city boring. Amsterdam used to be about sex, drugs and rock 'n' roll. Now we have nothing. No sex, no drugs and no rock 'n' roll."
And coffee shop owners have told Newsbeat they don't think the mayor has the power to close down smoking establishments.
"Coffee shops let people smoke in a civilised, respectable manner," said Michael Veling, who runs the 420 cafe just outside the red light district.
"We fill a demand from the market and keep people away from street dealers.
"In the coffee shop you see a peaceful coexistence between all the different groups that have come to Amsterdam in the last 50 years."