In the first public hearing on the designs, held in lower Manhattan on Monday evening and broadcast to locations across the city, the majority of people expressed dismay.
"
When we see the eighth wonder of the world we will know, and we certainly have not seen it yet
"
New York resident
Earlier blueprints for the site were abandoned by New Yorkers for being unimaginative and this time around, the competing companies were given more freedom with the designs.
Seven architecture companies submitted the nine different design concepts, most of which controversially feature buildings which would be even taller than the original towers.
On Tuesday evening a second public hearing will be held to debate what kind of memorial should be built at the site to commemorate the events of 11 September 2001 and those killed.
The question of what to do with the patch of prime real estate in the middle of New York's financial district is a controversial one with opinion differing wildly.
At one extreme are those who say no development is appropriate as the area is a mass grave. At the other are those who advocate building an even taller tower as an act of defiance against terrorism.
In the next few weeks the Lower Manhattan Development Corporation (LMDC) and the Port Authority of New York and New Jersey, which owns the site, will decide which of the designs gets the go-ahead.
It is expected that which ever one is selected will take about a decade to build.
Place in history
But on Monday evening it was evident that many New Yorkers were hoping that none of the plans submitted will be given the green light.
"I have followed the rebuilding debate from the beginning and what has transpired in the last 15 months has left me ashamed to be from this city," said one of the 700 people attending the hearing at Pace University.
"
I don't think that it's appropriate to build this way again
"
Peter Gadiel, father of WTC victim
The main complaint from New Yorkers seems to be that none of the designs provides an adequate memorial for those killed when the twin towers toppled.
They fail to capture the magnitude and significance of the events on that day, critics say.
Seeking the extraordinary
At the vanguard of these complaints is former mayor Rudolph Giuliani, admired around the world for his steadfast leadership during New York's darkest hour.
"After the design says to you, in a compelling way, this is one of the most significant places in the history of this country, then after that you can figure out how you fit in the office space and how you fit in everything else," Mr Giuliani said in December.
The city's residents are determined that what replaces the World Trade Center must be an architectural gem that will stand as a testament to 11 September.
Or as one person attending Monday's hearing said: "When we see the eighth wonder of the world we will know, and we certainly have not seen it yet."
Collective acrophobia
The second issue proving a sticking point is safety.
Five of the nine designs call for structures bigger than the world's tallest building, the Petronas Towers in Malaysia.
The thought of working on the upper levels of a skyscraper now strikes fear into the hearts of many New Yorkers.
In August 2002, a New York Times/CBS survey found that 53% of respondents said that they would not want to work in the upper floors of a new tower.
The issue has particular resonance for the relatives of those who died, trapped on the burning upper levels of the twin towers.
"I don't think that it's appropriate to build this way again," said Peter Gadiel, whose 23-year-old son, a worker at the World Trade Center, was one of the victims.
"If you really want to do this then have the Port Authority officers at the top of these buildings, have the federal officers the FBI or the CIA at the top of these buildings," he added.
"I am not concerned how tall the buildings are, I am concerned about how safe the buildings are," agreed Sally Regenhard, the mother of a firefighter who died.
As the debate rages on, the only thing that seems to unite New Yorkers is that the wound on New York's skyline must be healed with a dramatic building and that none of those produced so far comes up to scratch.