Experts say this delay could deny patients the chance of treatment - or to prepare for the onset of the progressive disease.
In addition, the survey, carried out by the Alzheimer's Society, revealed that the majority of GPs did not feel confident about either giving a diagnosis of dementia - or giving information about it to relatives.
Early Alzheimer's symptoms include memory problems, disorientation and confusion.
However, more than half of all current carers waited years before getting a medical opinion.
Professor Murna Downs, from the University of Bradford, who led the research, said: "Three years is far too long for someone to wait.
"We need to let people know that there are many new services and drug treatments for people with dementia."
There are drug treatments for dementia - but the earlier they are given, the more benefit there is for the patient.
Rising trend
The research focused on hundreds of patients and their carers, and was conducted over a three year period.
One in 20 people over the age of 65 and one in five aged 80 or over are affected by dementia - half with the most common form, Alzheimer's disease.
Harry Clayton, the chief executive of the Alzheimer's Society, said: "People with dementia are clear in what they want.
"Among the things they value are an accurate diagnosis given sensitively, prompt referral to specialist services, access to new treatments and good information about dementia and support services."
It is estimated that the total number of people with dementia will rise as the population ages.