But Mr Milburn rejected the suggestion that grammar schools should be re-introduced, saying that they might have worked when there were 250,000 university students but would not be relevant when there were 2.5 million.
The wide-ranging study by an independent panel of experts, Fair Access to the Professions, calls for more equal opportunities in education and employment.
It wants to raise the aspirations of more young people to have the expectations of professional families, giving them confidence when making decisions about university and jobs.
'Forgotten middle class'
The report warns that people entering careers such as medicine, law and journalism are increasingly likely to be from more affluent families.
Currently 75% of judges and 45% of senior civil servants are privately educated.
The report does not only focus on the poorest part of the population - but suggests that many middle-income families are also missing out in an increasingly polarised jobs market.
Mr Milburn warned of the "forgotten middle class" which could not compete with the advantages of the most privileged families, but which also did not benefit from the support targeted at the poorest.
The report also criticises informal recruitment systems, such as internships and work placement, as becoming a back-door for better-off, better-connected youngsters.
ITV executive chairman Michael Grade, a member of the panel, said the current internship system, based on "who you know", was "grossly unfair".
Another member of the panel, Major General David McDowall, said there needed to be more support for other ways of helping young people to develop ambition, through voluntary activities such as scouting or the Duke of Edinburgh awards.
The report recommendations include:
University students should be recruited from a wider range of social backgrounds, with no-fee degrees for students living at home
Higher education to be more widely available in further education education colleges
Universities to become more involved in schools, such as by having representatives on boards of governors.
Professions and universities to publish more details on the social background of their intake
Better careers advice raising pupils' aspirations
More extra-curricular activity for state school pupils, such as cadet forces
Help for parents to move their children out of under-performing schools
The report says much more needs to be done on university admissions - with fears that the university system can reinforce disadvantage rather than reduce it.
The efforts to widen participation had so far not been cost effective, Mr Milburn suggested.
There were also calls for an "acceleration" of school reform, such as increasing the number of academies and support for parents wanting to take their children out of failing schools - which Mr Milburn described as a "ghetto" of disadvantage.
The prime minister's official spokesman said Gordon Brown welcomed the report, which would get "a fair wind" in Whitehall, while Business Minister Pat McFadden said it provided "a welcome and rigorous examination of where we have done well and where we have fallen short".
He added: "We share the aim of the report's authors - to enhance the life chances of every young person regardless of their background or income."
David Willetts, Shadow Universities and Skills Secretary, welcomed the proposals on careers advice and school reform.
"We also welcome his suggestions on broadening access to university. The report would have been even better if it had tackled the need for strong families and better training opportunities," said Mr Willetts.
Lib Dem schools spokesman David Laws said finding the resources to tackle social mobility in the coming years would be a "challenge".
"Labour's tragedy is that on many measures Britain is less equal today than it was when Tony Blair was elected in 1997," Mr Laws said.
"Alan Milburn's report includes some useful proposals to improve social mobility - but the most important measures must involve early intervention and improved educational opportunities."
The British Medical Association (BMA) said it welcomed some of the report's recommendations, but said the government had restricted Mr Milburn's remit too tightly.
Tim Crocker-Buqueof the BMA said just 4% of medical students came from the bottom two socio-economic groups, adding: "Fee exemptions for students living at home will not solve this problem as most medical students do not live within travelling distance of the 32 medical schools in the UK."
Collette Marshall, Save the Children UK director, said the government "needs to close the educational attainment gap far earlier" if it wants to widen access.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.