There had been speculation the 40-year-old, a devout Catholic, could leave the government because of her objections to the Human Fertilisation and Embryology Bill.
Bolton West MP Ms Kelly, who previously served as communities secretary and education secretary, is understood to have wanted to resign in May, but to have been asked by Mr Brown to stay on until the next reshuffle.
BBC political editor Nick Robinson said Ms Kelly was known to be unhappy with the direction of the government in recent months and, before the prime minister's conference speech on Tuesday, a number of cabinet ministers had been considering resigning with her.
But Ms Kelly told the Labour conference in Manchester: "It's been a tremendous privilege to have worked with both Tony Blair and Gordon Brown, two towering figures in the Labour Party, government and on the world stage.
"But, as well as a frontline politician, I'm also proud to be a mother and a wife. To have been able to hold these jobs, I've relied on the support of my husband and my family.
"So I ask for your understanding when I say that I now owe it to my children and family to take a step back and start putting them first.
Gordon Brown gives his reaction to Ms Kelly's decision
"If I do not, then I know that this is something I will come to regret deeply."
Ms Kelly, who on Tuesday dismissed suggestions she might leave her job during an interview with BBC Radio Manchester, said: "This was not a decision I took lightly... The past 15 years have been an amazing and, at times, humbling experience."
Visibly moved when talking about her family, she was given a standing ovation by delegates. The Labour conference closed with a speech by deputy leader Harriet Harman.
The prime minister, touring the television and radio studios on Wednesday, paid tribute to Ms Kelly and said her decision - which was first reported on BBC Newsnight - was a personal one and "nothing to do with politics".
He told BBC Radio 4's Today programme: "There are no political issues between Ruth and me."
He also said: "Ruth has been an MP all the time that her children have been growing up...
"This is the decision that every parent faces. It is nothing to do with politics. Sometimes we have got to make decisions that are difficult."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.