The finding is at odds with previous research that has suggested that smoking may protect against Alzheimer's disease and dementia, both of which are associated with ageing.
A team from the Institute of Psychiatry in London carried out a community survey of an area of north London.
They studied more than 650 people aged 65 and over.
Smoking and drinking habits were recorded at the beginning of the study.
The volunteers' intellectual powers were also assessed.
After excluding those who had already had evidence of intellectual impairment, everyone else was followed up a year later to see whether there was any evidence of a decline in intellect.
Four times the risk
Of the 417 people who could be retested, one in 16 had suffered significant intellectual decline.
The researchers found that smokers were up to four times more likely to have evidence of significant intellectual decline than either non-smokers or former smokers.
This was after taking account of factors known to affect brain function in older people, such as depression and alcohol use.
Researcher Martin Prince said: "Our results indicate that persistent cigarette-smoking into late life increases the risk of cognitive impairment."
Smoking contributes to vascular disease and atherosclerosis, conditions which narrow and harden the arteries and impair blood supply to all parts of the body, including the brain.
The researchers suggest that this may explain why smoking has an impact on intellect.
The study also found that people who drank moderately before the age of 65 were marginally less likely to have a decline in their mental powers than either heavy or non-drinkers.