David Cameron has told an audience of rank-and-file police officers he has never urged people to "hug a hoodie".
Mr Cameron says he has been attacked for those "three words"
Speaking at the Police Federation conference, the Tory leader said a 2006 speech he made on crime was the most misrepresented thing he had ever said.
He told 1,000 officers at the Blackpool event that aggressive youngsters who threatened others should be punished.
But he added society needed to take responsibility for crime, rather than relying on the police alone.
Mr Cameron told officers he had been frequently attacked for "three words" he never said in his July 2006 speech on crime. That speech called on people to think before labelling teenagers in hooded tops "gangsters".
"So let me try again," he said.
"Aggressive hoodies who threaten the rest of us must be punished. They need to know the difference between right and wrong, and it's our job to tell them.
"I'm a Conservative. I believe in punishment, I believe in deterrence, I believe in the difference between right and wrong.
"But what do we really want, a society where more and more kids are out of control, a rising tide of crime and punishment? "Or do we want those kids to behave properly in the first place?"
Mr Cameron said that mending a "broken society" was not the police's responsibility.
"We ask a lot of the police," said Mr Cameron.
"But increasingly, I think, we ask too much. You don't need me to tell you how urgent a political priority crime and anti-social behaviour have become.
"But I believe that the entirely appropriate political prioritisation of crime and anti-social behaviour has, in recent years, given rise to a dangerous distortion.
"The perception [has built] that fighting crime is simply a question of law enforcement and policing. And that if crime goes up, the police are to blame.
"The reality is different. Crime, anti-social behaviour, disorder and incivility on our streets: these are the consequences of a breakdown in society - of a collapse in social responsibility."
"We broke our society - all of us, as parents, as citizens, as members of society - we broke it, and we have a shared responsibility, with government, for fixing it," he said.
"If we sit back and expect the police to do all the work, we will forever be managing the social problem of crime, rather than solving it."
Mr Cameron claimed the government had burdened police officers with targets, which had taken away their individual discretion.
Earlier in the conference, Home Secretary John Reid denied that charge, saying that he had set only three targets focused on cutting crime and improving public confidence.