In an interview for Newsbeat and BBC Two's Newsnight, the Tory leader also defended his choice of wearing floral shorts while playing Frisbee on the beach in Cornwall.
He said: "I think my wife bought them, I think they're quite nice.
"They're Boden, that catalogue company. She said you need some shorts and she takes the major decisions, of course.
"They're all right aren't they? What, a bit flowery? I've had them for years."
David Cameron and wife Samantha are on a week's break
Mr Cameron said he was on the beach when he noticed the photographer taking a picture of him wearing them.
"I realised something was up - but luckily I caught the Frisbee so it wasn't too bad," he said.
Asked by another listener whether he was smug about Gordon Brown's recent battering in the polls, Mr Cameron said: "I hope I'm not smug because I've said I don't want us to win just because Labour are failing, I want us to win because we deserve to.
"And I don't think we would deserve to win unless we can convince people... that we've got better ideas."
He added: "I think Gordon Brown has probably helped me by making bad decisions and that's good for me but I want people to think when they go to the polls, 'Right, David Cameron, I know what he thinks about schools, about welfare, about the NHS...'
"I've still got a lot of work to do to convince people that I'm the right person."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.