Critics have warned wearing the vests will encourage attacks
Offenders have been abused and threatened because of high visibility vests worn during community service, the probation officers' union has said.
The union also said many groups that ran community service projects, such as churches and charity shops, had refused to make offenders wear them.
The vests, marked with "Community Payback", were introduced for offenders in England and Wales on 1 December.
The government hopes the vests will raise respect for community sentences.
In one area a group of youths shouted "nonces, smackheads, lowlifes" at a work group, in another offenders were sworn at and in a third case cans were thrown, said the National Association of Probation Officers (Napo).
Harry Fletcher, assistant general secretary of Napo, told the BBC about 35,000 people were on community service projects at any one time.
He said that as of 19 December two thirds of those projects had received the vests and of those there had been about a dozen incidents of abuse.
Mr Fletcher said it was possible to veto whether offenders wore the vests, but that this was not down to probation officers but the private groups that host the community service work, such as churches and charity shops.
He went on to say that in one area of the Midlands, 28 out of 32 placements had refused to give offenders the vests and in another area in north-east England, 11 out of 20 had rejected them.
"Refusal has normally been on the grounds that it demeans the individual, introduces unnecessary risk, or deters customers from charity shops," said Mr Fletcher.
"There is absolutely no evidence that the wearing of vests has any impact whatsoever on crime. Indeed the early trends suggest that more offences will be committed as a consequence of the jackets being worn."
But Justice Secretary Jack Straw defended the use of the vests, saying that if there were genuine concerns for the safety of staff or offenders in particular cases, consideration would be given as to whether the jackets were appropriate.
Speaking to the BBC when they were introduced, he said: "The purpose of having these high-visibility jackets is, above all, to strengthen the confidence of the public in community punishments because too few of the public believe at the moment these are effective and are other than a soft option."