More than 15,000 people are believed to have been killed in drug-related incidents in Mexico in the last two years.
There were more than 2,600 murders in Ciudad Juarez last year.
On 31 January, 13 teenagers and two adults were shot dead at a high school party. Their families have said they had no gang ties.
"If those deaths... mean anything it is that we need to change after that absurd sacrifice," Mr Calderon told Ciudad Juarez residents.
The president promised improvements in health, education, welfare and infrastructure for the poverty-stricken northern city, but insisted he would not withdraw the 6,000 troops deployed there.
Julian Miglierini, BBC News, Mexico City
This was a highly unusual day in Mexican politics.
In a country where the institution of the president is deeply respected, Felipe Calderon exposed himself during several hours to direct accusations of inefficiency in his strategy to combat the drug cartels.
The people of Ciudad Juarez vented their frustration with the ongoing violence and the effect it is having on daily life in the border town where more than 5,000 people have been killed since 2006.
Mr Calderon said he was ready to readjust his policies and launched a series of social initiatives, but insisted he would not pull troops out, one of the popular demands.
His visit was aimed at opening a new chapter in the fight against drug trafficking, but as one of the local leaders told Mr Calderon to his face, it may have come too late.
"I've promised the parents of the victims that we'll find a new impetus for the fight against the violent gangs," he said. "We have to have better co-ordination between the different institutions of government and the police forces to take on this challenge - a fight that we have yet to win."
But as soon as he finished his speech and opened the floor to questions, the criticism started to flow, says the BBC's Julian Miglierini in Mexico City.
"You come here one or two years late," a local leader told Mr Calderon, while a woman shouted: "You are not welcome here."
All those who spoke expressed their frustration about the level of violence, our correspondent adds.
They also complained about the lack of a proper infrastructure and alleged human rights abuses by the security forces.
The federal government's perceived lack of efficiency in dealing with the crisis was another major grievance.
Army helicopters patrolled the skies above, as federal riot police tried to disperse the dozens of people staging a demonstration.
Many were holding signs saying "army and police, leave!" and "Calderon out".
"Calderon's afraid... We want justice, we want him to resign," Luz Davila, whose two teenaged children were killed in the January high school party shooting, told the Reuters news agency.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.