The break-up of co-habiting couples leads to severe social problems, a Conservative report has warned.
Family relationships need to be strengthened, Mr Duncan Smith said
Half of co-habiting parents split by their child's fifth birthday, while only one in 12 married couples did, the social justice group report says.
It puts the cost of this family breakdown - in poverty, drug abuse and debt - at £20bn a year.
Iain Duncan Smith's group, which is shaping Tory policy on poverty, publishes its interim report next week.
The report, Breakdown Britain, will also warn of the collapse of what it calls "the welfare society", the informal support network within families and communities.
Mr Duncan Smith - a former Conservative leader - said: "What we found was one of the fastest growing groups that are having children in society are co-habiting parents but what was startling about the figures that showed as co-habitees one of two of them is going to break and become a single parent household before the child is five."
He said the consequences were that "the state picked up the pieces", as single mothers were left bringing up children on a greatly reduced income.
"We need to ask ourselves what's going on why do we have such high levels of family breakdown," he said.
He insisted that the focus of the report was not to "lecture" people to get married, but to help couples, both married and co-habiting, to stabilise their relationships.
"We can't have a stable workforce and a productive economy if a growing number of people at the bottom end of society are workless, and without hope," he said.
Mr Duncan Smith said he was not making a moral judgement about marriage.
Last month, Tory leader David Cameron claimed there had been a "big change" in the Conservatives think about poverty and that it existed in "relative" terms.
In his speech, Mr Cameron said: "I believe that poverty is an economic waste, a moral disgrace.
"So I want this message to go out loud and clear - the Conservative Party recognises, will measure and will act on relative poverty."