"I think it would be a good idea to have a new evaluation when we reach 70, then 75, 80."
There are over three million licence holders over the age of 70, and that number is projected to reach 4.5m by 2014.
Currently elderly drivers have to declare their medical conditions to the DVLA when they reach 70.
The Department for Transport said it is planning a consultation on health and the driver licensing system.
Sir Stirling's concerns are shared by Madeline Selwyn, from Aberdare, Rhondda Cynon Taf, whose 19-year-old son Jonathan was killed in March 2002 when his car was hit by a 69-year-old travelling the wrong way down the M4.
Mrs Selwyn said this changed the way she views elderly motorists.
"It's funny," she said, "I look at people as I pass them, rather than their cars.
"I think we've all seen people getting out of cars and they can hardly stand or walk. You think, 'should they be driving?' It can be frightening sometimes."
According to the Department of Transport older drivers are set to become more numerous and are predicted to be driving further and more often.
Week In Week Out, Tuesday 23 June, BBC ONE Wales, 2235 BST
Leslie Philips, 76, says over 70s shouldn't be forced to give up their driving licenses and that a voluntary system would be more appropriate
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.