David Cameron has come under fire from the home secretary for saying there was "anarchy in the UK" following the killing of 11-year-old Rhys Jones.
Jacqui Smith said the fight against violent crime was not helped by those who sought to make political advantage out of such incidents.
Addressing the TUC, she said crime was falling, but the fear of crime was having a damaging impact on people.
Mr Cameron has stood by his comments and denied he was "scare-mongering".
He said the killing of 17 teenagers in London this year illustrated the "very serious breakdown of order in some of our communities".
In her speech, Ms Smith said fear of crime was weakening the social fabric of society and breeding mistrust.
"The situation is not helped by those who seek political advantage from unnecessarily shrill warnings that we are facing anarchy on our streets," she said.
"Of course, tragic events affect us all and rightly make us question whether we're doing everything we can to tackle crime. But Britain isn't broken," she said.
She told delegates in Brighton that burglary was down by 60% since 1995, while violent crime was at its lowest level for a decade.
The likelihood of being a victim of crime was at its lowest level for 25 years, she said.
"These aren't the symptoms of a broken society - and to suggest otherwise is not only wrong, it blurs the issue."
But shadow police minister David Ruffley accused Ms Smith of being "in denial".
"The government's own statistics show that violent crime has doubled under Labour while gun-related violence has gone up four-fold," he said.
"This dangerous complacency on behalf of the government is putting the public at serious risk."
At the weekend, Ms Smith announced that a specialist national police unit and a ministerial task force will be set up to try to tackle gun crime in major English cities.
She said she was worried gun crime was "happening more with young people".
The £1m initiative will target gun crime and gangs in London, Manchester, Liverpool and Birmingham.
Rhys Jones was shot as he walked home across a pub car park in Croxteth, Liverpool on 22 August. His funeral was held last week.