Funding should be taken away from underperforming further education (FE) colleges and given to better performing institutions, MPs say.
The MPs say colleges must be more attuned to the needs of employers
The Public Accounts Committee (PAC) says 35 colleges offer unsatisfactory provision and the taxpayer should not have to support "continuing failure".
The committee criticises the FE sector as complicated and over bureaucratic.
The findings follow a report published in November by Sir Andrew Foster which found 10% of colleges were poor.
Sir Andrew's study, commissioned by the government, said a "a residual rump" of about 10% of colleges should have a deadline to improve or face takeover by another college or outside body.
Now the PAC has examined the progress made in improving the arrangements for planning, funding and monitoring expenditure on learning and skills.
Sir Andrew Foster also highlighted weaknesses in the sector
The committee wants local learning and skills councils to shift funding to more effective colleges or to support mergers with "better managed colleges".
MPs also raise concerns about the levels of bureaucracy in the sector.
"With 500 organisations involved in the planning and delivery of further education, there is significant risk of duplication and wasted effort from overlapping roles," the report says.
It says 397 colleges, 47 local Learning and Skills Councils, nine Regional Development Agencies, nine Regional Skills Partnerships and 25 Sector Skills Councils looks like "excessive bureaucracy".
Structures must be simplified so less money is spent on administration.
The figure of 35 colleges that were unsatisfactory came from evidence given to the committee by the chief executive of the Learning and Skills Council, Mark Haysom.
The council said later the proportion of colleges judged inadequate by Ofsted was down to 2%.
"This means that only nine FE colleges are currently classed as inadequate - and we are working hard with them to effect change," said director of strategy and communications, Rob Wye.
He added: "It is true that a very small number of colleges are failing to pass their inspections. But hundreds others are succeeding and are providing quality service to their local communities."
Skills employers want
Concerns are also raised in the MPs' report about the lack of consultation colleges have with business and employers.
"Colleges will have to offer the courses which employers and learners want and for which they are prepared to pay," said Edward Leigh, chairman of the PAC.
"They will therefore have to be more attuned to learners' career aspirations and the needs of business.
"They will also have to develop workable fees policies to help make less commercial courses sustainable."
The Association of Colleges said it supported the call for more funding to be diverted away from bureaucracy towards the delivery of education and training.
"There is clear evidence that the current systems used to plan, fund and regulate our colleges are complicated, costly and ineffective," said AoC chief executive Dr John Brennan.
Higher Education Minister Bill Rammell said the report raised issues the government was considering in response to Sir Andrew Foster's report.
"We intend to publish a White Paper in the following weeks outlining how we will address the shortcomings that currently exist within the FE sector and deliver a world class system which will not only drive up skills but raise our economic prosperity," he said.
Red tape would be cut, duplication reduced and poor performance tackled.
Liberal Democrat spokesman Tim Farron said modern, efficient, well-managed colleges could solve Britain's "looming skills crisis".
"But that cannot and will not happen without urgent reform," he said.
"The current system is a mess and good colleges are being hamstrung by red tape and bureaucracy."