"When I was growing up in the Midlands, the rocks and limestone you found were full of the most magical things," he recalls.
"You hit a stone and it suddenly fell open, and there was this amazing coiled shell, beautiful and extraordinary, and nobody had seen that for 150 million years, except you."
He found the experience romantic and exciting.
And it appealed to his small boy's instinct of collecting things.
I wanted to go back to the beginning
Sir David Attenborough
"To be honest I don't think I've really lost that," he says.
So it seems appropriate that his latest series, First Life, explores the very origins of life on Earth.
For a broadcaster now age 84, he reveals that First Life is actually a long-awaited first chapter: one that he had not yet had the chance to visit during a long and illustrious career.
"We always started at chapter two," he says during another documentary Attenborough's Journey, screened on Sunday on BBC Two which followed Sir David as he travelled the globe to film the First Life series.
"I wanted to go back to the beginning and today there's real evidence of how life on Earth started."
During First Life, he travels from Newfoundland to North Africa and the rainforests of Queensland, looking for evidence in both fossils and living animals there of a period in Earth's history half a billion years ago, when animals first appeared in the oceans.
Attenborough's Journey provides a preview of the series, but also an insight into mind of the presenter and highlights from his own life.
Sir David is captured on camera haggling with a fossil salesman in Morocco over trilobite fossils, and pacing back and forth as he prepares and practises his lines in his inimitable style.
In candid interviews, he recalls some career-shaping moments, including the first documentary he presented for the BBC called Zoo Quest, which was first broadcast in the 1950s.
For that series, he actually undertook an animal-catching mission - travelling the world with staff from London Zoo with the aim of bringing back creatures for the zoo's collection.
A memorable encounter with a gecko
During the programme he explains how he brought some of the animals home, turning the Attenborough household into something of a menagerie.
Return to adventure
A very successful early career saw Sir David promoted to controller of BBC Two while he was still in his 30s.
But, ever the adventurer, he says that he was never happy behind a desk. He resigned from this post and went on, almost immediately, to make the Life on Earth series that cemented his long partnership with the BBC Natural History Unit.
That partnership continues: one of the First Life series team says of Sir David during the film that he cannot imagine this presenter, writer, naturalist and inspiring communicator ever retiring.