The study for the TUC found that 30% of workers said fear of violence was among their top five concerns.
Stress was the main concern, followed by repetitive strain injury, the effect of screens and back strain.
The report, published on Saturday, was based on the experiences of 5,000 union-appointed safety representatives.
Top Five Concerns
55% Overwork and stress
37% Repetitive strain injury
34% Display screen equipment
31% Back strain
30% Fear of assault
It was the first time violence had figured so highly, having been placed eighth in the last poll in 2000.
Employees working with the public, such as jobcentre staff and hospital workers, are most at risk.
John Monks, general secretary of the TUC, said: "The increase in worries about violence, which reflects actual increases in the number of assaults on workers, is especially troubling.
"Too many workers face the threat of violence when they go to work and in some jobs, the only question is 'when' will you get attacked not 'whether'.
"Workers are facing a rising tide of violence and employers haven't got to grips with the threat.
"Individual acts of violence are random but violence can and should be assessed, managed and reduced."
Strike over safety
Earlier this week, a poll of 5,000 Department of Work and Pensions staff, including many working in jobcentres and Benefit Offices, found that 49% had personally experienced violence at work in the last year.
The survey by the Public and Commercial Services Union suggested the figure rose to 76% for those working directly with the public.
The union went on strike to protest about new open plan offices in jobcentres.
More than a quarter polled said they expected to be assaulted in the next year.
The TUC said it is working with the Health and Safety Executive, the Home Office, unions and employers to spread good practice to prevent violence at work.
A conference is to be held at TUC headquarters in London on 2 December.
There will be a breakdown of regions and sectors where violence and threats are said to be a problem released at a later date.
Another survey earlier this year revealed that more than half of all office staff in the UK have become so angry at work they have nearly punched a colleague.