It also argued ministers needed to work more closely with a number of different organisations to ensure objectives were met, especially within those groups most at risk.
The report admits the reasons are complex but says cyclists are most at risk of being killed by lorries and children are more likely to die on the roads after school.
Darker evenings after the clocks go back are also a factor.
National Audit Office head Tim Burr said: "Making roads safer for pedestrians and cyclists is a key element in encouraging people to walk and cycle more.
"While their safety has improved generally, some are more vulnerable, such as child pedestrians from deprived areas."
In the report, Britain's 2006 and 2007 road death figures for pedestrians and cyclists were compared with those from around the world, as well as with those from the developed countries in the Organisation for Economic Cooperation and Development.
In 2007, 2,956 people were killed on Britain's roads. Of those, 646 were pedestrians and 136 cyclists.
The audit office found that, compared with figures for the mid-1990s, there had been a slower rate of decline in road deaths, down 18%, than serious injuries, which dropped by 37%.
It also found that while the number of cyclists killed or seriously injured on Britain's roads fell between 2000 and 2004, they rose again by 11% between 2004 and 2007.
Kate Cairns, whose sister was killed on her bike, talks about the vulnerability of cyclists
In 2007, 136 cyclists were killed, while more than 16,000 were injured.
Edward Leigh, chairman of the House of Commons Public Accounts Committee, said Britain had a good record of reducing road deaths overall, but needed to do more to cut those among pedestrians and cyclists.
"As a father of children, this concerns me greatly. Our relatively poor performance should be a matter of national debate," he said.
A DfT spokeswoman said the department welcomed the audit office's findings and was addressing the points raised.
"The number of child pedestrians aged 0-15 killed on Britain's roads has fallen by 57% since the mid-1990s, showing that the work of local authorities, the emergency services and national government is making a real difference," she said.
"This is a dramatic reduction but we are not complacent and that is why we have set a target to reduce the number of deaths and serious injuries on our roads among children aged 0-17 by 50% by 2020."
The government's Think! road safety campaign and new website also offered information and educational materials to children, teachers and parents, she added.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.