The Sun newspaper had an exclusive interview with the parents
The case of the 13-year-old who has fathered a baby highlights a worrying trend of "children having children", the Tory leader has said.
David Cameron was speaking after Alfie Patten's girlfriend Chantelle Steadman, 15, gave birth on Monday.
Latest figures from UK National Statistics show there were 7,826 conceptions in under-16s in 2006 - down 1,000 from 10 years earlier.
Sexual health experts said teens should be encouraged to have "ambition".
Children's Secretary Ed Balls said it was an "awful" and "unusual" case.
"I want those kids to be safe and also the young child as well, and I want us to do everything we can as a society to make sure we keep teenage pregnancies coming down," he said.
Mr Cameron said: "It is really worrying in our country today you've got children having children - and obviously we all hope that these very, very young children will grow up and be good parents, but frankly parenthood isn't something they should be thinking about.
"That's what's gone wrong and we've got to put it right."
Former Tory leader Iain Duncan Smith, who now runs the Centre for Social Justice said it "exemplified the point we have been making about broken Britain".
He told the BBC the problem of teenage pregnancies was concentrated in certain communities.
"Quite often what fails to recognise is that in some of the more difficult communities, this has become pretty much the norm.
"I've visited many estates where there are very huge collections of very young mothers, often with multiple children - and often 'guesting fathers'.
"This is a major problem in the UK."
The teenagers, both from Eastbourne, had kept the pregnancy secret until Chantelle's mother Penny noticed her daughter was getting bigger.
Chantelle had her baby in Eastbourne District General Hospital, and is now home with her family.
Sussex Police said no action would be taken with it "not in anyone's interests" to prosecute.
East Sussex County Council said the teenage parents will be supported with intensive monitoring and health visitor support.
Tony Kerridge, a spokesman for the sexual health specialist Marie Stopes International, said: "These sorts of lifestyle choices can be dealt with on an educational level if teenage girls realise what they are contemplating is a route into social deprivation and being in the benefits culture for the rest of their lives.
"It may seem like a short term solution to problems at home, but the mid to long term prospects are probably a life stuck on benefits.
"We should, as a society, be encouraging our young people to have much bigger and better aspirations than that."
Simon Blake of Brook added: "Most under-16s aren't having sex, but young people do want more education about sex.
"They want their parents to talk to them, their school to talk to them and they want confidential services to go to."
Speaking on a visit to the Midlands, Prime Minister Gordon Brown said: "I don't know the individual details of the case, but of course I think all of us would want to avoid teenage pregnancies."
Beverley Hughes, children and young people's minister, said 80% of under-18 pregnancies were in 16 and 17-year- olds.
"Evidence shows that good quality sex and relationship education and parents talking to their children delays early sex and increases the use of contraception whenever they do have sex.
"We know that life as a teenage parent is hard, with outcomes for them and their children often very poor.
"That's why the government has issued guidance to local areas on providing the right support to improve the welfare of both the teenage parents and child."