Lord Carter and Sir David are the latest to comment on the controversy
The BBC was "unacceptably slow" to respond to the Russell Brand-Jonathan Ross calls row, broadcasting minister Lord Carter of Barnes has said.
However, he said "the BBC, its Trust and the industry regulator are now all doing what they are there to do."
Lord Carter, a former chief executive of media regulator Ofcom, was replying to a debate on a Lords committee report on ownership of the news.
His comments come in the wake of the recent lewd phone calls row.
The BBC Trust is expected to raise the issue at its Editorial Standards Committee meeting later.
Ross received a 12-week suspension for his part in the row, which involved offensive messages left on the answer phone of Fawlty Towers actor Andrew Sachs.
Brand resigned from Radio 2 as a result of the incident, as did station controller Lesley Douglas.
Ross's suspension will not stop the star filming an unpaid appearance in a celebrity edition of The Apprentice, to be aired as part of next year's Comic Relief broadcast.
"The BBC did not wish to disadvantage the charity which Jonathan has supported for over 20 years," said a spokesman.
"The show will not be broadcast until March, well after the end of his suspension in January."
Sir David Frost has become the latest personality to comment on the controversy, saying it was "inexplicable" that the lewd phone messages were aired on Brand's Radio 2 show.
"I don't understand how this ended up being broadcast," he is quoted as saying in The Sun. "There didn't seem anything humorous about it at all."
However, the veteran broadcaster said he hoped there would not be a backlash against satirical comedy.
"I'd hate for this incident to mean we start going backwards," he added.
Stephen Carter was appointed junior minister of communications, technology and broadcasting in October.
The former Ofcom executive was made a life peer at the same time.