Mr Hunt said BBC employees tended to hold centre left political views
The shadow culture secretary has called for the BBC to "actively" recruit more Conservatives to its news team.
Jeremy Hunt said the BBC had "made huge strides" on diversity issues but should "not forget the core audience".
The BBC said it would never recruit people for their politics and Culture Secretary Ben Bradshaw described it as "unacceptable political interference".
Mr Hunt made the comments after he was asked about the age row and the BBC's hunt for an older female newsreader.
He was questioned by reporters about whether he thought the BBC was institutionally ageist following the row over the axing of Arlene Phillips from Strictly Come Dancing.
Mr Hunt said: "I do think they might need to do more for older viewers.
"The BBC tries very hard to do the right thing. In terms of diversity issues it's made huge strides, but it's important it does not forget the core audience."
Mr Hunt said he did not want to comment on whether removing Phillips was "ageism".
But he later added: "I wish they would go and actively look for some Conservatives to be part of their newsgathering team."
He said it was acknowledged people who want to work at the BBC tend to be from the centre-left.
Mr Bradshaw responded: "This is unacceptable political interference in the BBC by the Tory Party.
"There are many Conservatives at the BBC but its journalists, whatever their private political views, are subject to strict rules of impartiality, which the BBC has a duty to enforce."
A BBC spokeswoman said: "Clearly we do not recruit people on the basis of their political views. We have not, and would not, recruit people in this way.
"Accuracy and impartiality are at the heart of BBC News and what's key to us is that the people working for us produce and deliver high-quality news that our audiences trust and value."
Earlier it emerged BBC director general Mark Thompson asked news director Helen Boaden to find an older female newsreader to counter accusations of ageism.
The BBC has recently denied accusations it was ageist after replacing Phillips, 66, with former winner Alesha Dixon, 30.