Even those that do buy insurance could be losing out by purchasing single-trip insurance from travel agents rather than shopping around.
In addition, the survey of 1,000 holidaymakers, sponsored by Liverpool Victoria the UK's largest friendly society, found that many are unaware of what the policies cover them for.
Of those people with travel insurance, 65% did not think that it covered them for personal possessions away from home.
Shop around
Half the travel insurance bought covered only a single trip, and was often bought from travel agents at higher premium rates.
The average cost of single trip travel insurance was £31.15.
However, with annual travel insurance, which offers cover on a multi-trip basis, the average premium was £59.20.
"Many people 'blindly' take single trip travel insurance from travel agents when buying their holiday, despite it being expensive," Andy Marsh, Household and Travel Underwriter, at Liverpool Victoria told BBC News Online.
Warning
The survey follows a warning from the UK government last week that many UK holiday makers taking short breaks abroad are leaving without travel insurance.
Figures from the Foreign Office showed that 43% of British travellers often failed to take out proper cover.
The government blamed the huge growth in the number of travellers taking advantage of low cost air travel and late holiday deals.
~RS~a~RS~International~RS~q~RS~~RS~z~RS~12~RS~)