Ms Smith was at Downing St for the weekly cabinet meeting on Tuesday
Jacqui Smith is to stand down as home secretary in the cabinet reshuffle, sources close to her have told the BBC.
Prime Minister Gordon Brown is set to shake up his cabinet after Thursday's European and English local elections.
The source said she was stepping down for her family, who had been "at the forefront" of a row over her expenses.
Ms Smith, who wants to remain an MP, was criticised for listing her sister's London house as her main home - and her husband's claim for an adult movie.
It is understood Ms Smith, the first woman home secretary, intends to defend her Redditch seat at the next election.
It comes as Labour backbencher Ian Gibson is told he cannot stand again for the party over his expenses claims and three other MPs announce their intention to stand down at the next general election.
Cabinet Office minister Tom Watson, a close ally of Mr Brown, is also expected to stand down from the government in the reshuffle - expected in the next week.
Mr Brown confirmed to the BBC he is planning a reshuffle but refused to be drawn on individual ministers' roles, amid speculation that Chancellor Alistair Darling, whose expenses have also been questioned, may also be moved.
The BBC understands that Ms Smith, 46, told Mr Brown during the Easter recess that she wanted to step down as home secretary.
A source close to Ms Smith said she was quitting her cabinet job because it was the "right thing for her family".
The source told the BBC that the row had put pressure on her children and her parents and while she regretted wrongly submitting a claim for the adult movie, she felt "vindicated" in her overall approach to claims, now those of other MPs had been published.
She made no mention of her plans while moving the Borders, Citizenship and Immigration Bill in the Commons on Tuesday afternoon but received several "hear hears" as she stood up.
But her opposite number Chris Grayling suggested it was her "final appearance" as home secretary and said she had "pre-announced the reshuffle".
She was promoted to the job when Mr Brown became prime minister in 2007 and initially won plaudits for her handling of the car bomb attack on Glasgow Airport.
But she has since come under pressure over issues - including attempts to extend pre-charge detention limits for terrorist suspects and the Home Office leaks inquiry that led to the arrest of Tory MP Damian Green.
Weeks before the Daily Telegraph began its revelations about MPs' expenses, Ms Smith's own claims came under question.
She had designated her sister's home, where she stays when she is in London, as her main home - rather than her constituency home where her family live.
Later she agreed to pay back allowances claimed for pay-per-view television services, which included two adult films apparently watched by her husband.
Labour's deputy leader Harriet Harman, interviewed on BBC Radio 4's World at One, said she could not confirm the "speculation" about Ms Smith.
But she added: "I think she is an outstanding home secretary. And yes there has been controversy around her expenses and she's not alone in that.
"On any side of the House there's been controversy but I don't think she's ever wavered from her commitment to her job as home secretary."
Chris Huhne, the Liberal Democrats' home affairs spokesman, said it was "extraordinary" that leaks suggested Alistair Darling was also "on the way out".
He said: "For a government to lose both the chancellor of the exchequer and the home secretary, two of the top four people, in one reshuffle does suggest that the real problem isn't the top four, it's the person right at the top, it's the prime minister."
Hewitt and Hughes
Mr Darling and Transport Secretary Geoff Hoon have also found their expenses claims questioned and have paid back some money claimed on their second homes.
Earlier BBC's political editor Nick Robinson asked the prime minister if both should remain in the cabinet. Mr Brown said so far people only had "the newspapers' version" of much of what had been claimed.
MPS LEAVING PARLIAMENT
The following MPs have said in the past three weeks that they will not contest the next election
Conservative: Andrew MacKay, Julie Kirkbride, Douglas Hogg, Sir Peter Viggers, Anthony Steen, Sir Nicholas and Ann Winterton, Christopher Fraser
Labour: Margaret Moran, Ben Chapman, David Chaytor, Ian McCartney, John Smith, Patricia Hewitt, Beverley Hughes, Michael Martin (Speaker)
"We're doing a far more extensive examination of what's happened with expenses and everything else. If there's a reshuffle, that's a matter for me about people's competences, ministers as well," he said.
"If any mistakes have been made and if they were exposed - then people will have to accept the consequences".
Communities Secretary Hazel Blears has also faced widespread speculation that she will be axed as a result of controversy over her second home claims.
Bury North Labour MP David Chaytor, who is accused of claiming for a mortgage that was already paid off, said that he would not stand at the next election.
Also on Tuesday, former Health Secretary Patricia Hewitt and children's minister Beverley Hughes announced they would stand down at the next election, but said it had nothing to do with the expenses furore.
Ms Harman said Ms Hughes and Ms Hewitt were going due to family reasons and rejected suggestions that the "wheels are falling off" the government.