Prince Charles's communications chief has defended his boss saying comments about schools were "misrepresented".
Charles Clarke warned the Prince to think before intervening
Education secretary Charles Clarke called the Prince "old fashioned" after he said the "learning culture" gave people hope beyond their capabilities.
"We can't all be born to be king," Mr Clarke told the BBC, insisting children should aspire to do their best.
But Paddy Harverson said the Prince was simply arguing against the idea of a "one size fits all" education system.
'Hardly dare say anything'
Tony Blair said it was an issue he would prefer to stay away from.
The Prince of Wales meanwhile responded to Mr Clarke's remarks by saying 12 years ago he was ridiculed for views he had expressed on tourism and the need for environmentally friendly architecture.
"But now people realise that old-fashioned views are coming round again.
"Perhaps my fiendishly old-fashioned views of 12 years ago are not so old-fashioned now."
He added "I hardly dare say anything. I don't really want to teach any more grandmothers to suck eggs."
Earlier, Mr Clarke accused the Prince of not understanding what happened in schools after remarks by the heir to the throne were published during an employment tribunal involving former staff member Elaine Day.
Miss Day had complained to the Prince about prospects for her promotion.
The Prince responded with a handwritten note saying: "What is wrong with everybody nowadays?
"What is it that makes everyone seem to think they are qualified to do things far beyond their technical capabilities?"
He goes on to blame the "learning culture in schools" and a "child-centred system which admits no failure" and tells people they can achieve greatness without "putting in the necessary effort or having the natural abilities".
Pressed on the issue on BBC Radio 4's Today Programme, Mr Clarke said: "I do believe it's very, very important that every child has the ambition for themselves to achieve whatever they can do for themselves - that everyone has a field marshal's baton in their knapsack."
"We can't all be born to be king but we can all have a position where we really can aspire for ourselves and for our families to do the very best that we possibly can and I want to encourage that position."
The education secretary said he did not want to get into a "tangle" with the Prince but went on to criticise him for speaking out.
"To be quite frank I think he is very old-fashioned and out of time and he doesn't understand what is going on in the British education system at the moment," he said.
"And I think he should think carefully before intervening in that debate.
"The key point which I think is so, so damaging is when whole groups of people are dismissed as having no possibility, no ambitions, nothing can be done with them. I think that is really damaging."
The Prince's communications secretary, Paddy Harverson, told BBC News that the heir to the throne's comments were misinterpreted.
He said: "I think where the misrepresentation was, was about people suggesting he was talking about sociology and social opportunity, he was talking specifically
about education," he said.
Shadow education secretary Tim Collins said Mr Clarke was unwise to "speak in the way he did".
He defended Prince Charles saying he was very committed to young people, particularly disadvantaged ones, and it was important that he felt able to speak freely on the issues that he cared about.
There was a difference between encouraging children to do the best they can and suggesting that all of them could get A grades in exams, Mr Collins said.
Downing Street earlier said it saw no need to apologise to the Palace for Mr Clarke's remarks.