Tory leader David Cameron has said his party must do more to keep families together, as a report suggests parental splits are creating an underclass.
Mr Cameron said stable couples benefited society
Couples should be "encouraged to get together and stay together" - possibly with the help of changes to taxation to support marriage, Mr Cameron said.
Iain Duncan Smith's report for the Tories on poverty, found family splits caused social problems costing £20bn.
An underclass was being "left behind" by the rest of society, he warned.
Launching the Breakdown Britain report in central London today, Mr Duncan Smith said some of the issues raised might cause party colleagues to "swallow hard".
But he insisted governments could no longer rely on the "small ships being lifted on the rising tide" theory, which suggests the poorest in society have their situation improved by increases in overall wealth.
"Some ships will never float - they've got a hole in them," he added.
Mr Duncan Smith said the prison population had spiralled in the last 15 years.
Typical inmates were young men, three quarters had a drug or alcohol problem and 60% came from broken homes.
Most had the educational level of a 10 or 11-year-old.
Mr Cameron said the report "demonstrates clearly the links that exist between social breakdown, family breakdown, educational failure, indebtedness and drug and alcohol abuse".
He added: "Families matter because almost every social problem that we face comes down to family stability.
"If marriage rates went up, if divorce rates came down, if more couples stayed together for longer, would our society be better off? My answer is yes."
Mr Duncan Smith rejected criticism for telling the Sunday Telegraph that same-sex couples were "irrelevant" to his work on shaping Conservative policy.
He told the BBC that he had not been making a value judgement, but merely reflecting the fact so few children were raised by gay couples.
Stability and structure was vital, he said, so "if they are bringing them up well, well done and good luck to them".
Mr Duncan Smith said marriage had been undermined by the tax and benefits system under the current government.
Mr Cameron told the Daily Mail he believed marriage should be supported through the tax system, such as through transferable allowances.
He has previously also indicated he favoured similar tax allowances for same sex couples in civil partnerships if they have children.
Work and Pensions Secretary John Hutton said it sounded like the Conservatives were going "back to basics again" - as in the mid 1990s - and talking about Victorian values.
He said it was "nonsense" to suggest that tweaking the tax system could lead to families staying together, saying that had been tried in past decades which saw divorce rates "go through the roof".