Prime Minister Gordon Brown has been on holiday in Suffolk
Downing Street has refused to comment on newspaper speculation that Prime Minister Gordon Brown is planning a cabinet reshuffle in early September.
But it did dismiss suggestions cabinet ministers have been told to return a month early from summer recess.
Parliament resumes in October, and Mr Brown has been on holiday in Suffolk.
BBC political correspondent Adam Fleming said there was consensus in the Labour Party that Mr Brown had to create a "big bang" on his return.
He said party members wanted Mr Brown to "re-assert his authority and help his party out of the hole they have managed to dig themselves into".
The Independent newspaper said on Saturday that all ministers were being told by Downing Street to return from holiday early next month, and said this signalled a wide-ranging ministerial reshuffle.
It said Foreign Secretary David Miliband - who recently wrote an article which led to speculation he wanted to succeed Mr Brown - would not be sacked.
Some advisers were instead urging the prime minister to "lock" Mr Miliband into his top team by making him chancellor of the exchequer, it said.
The Sun also says cabinet ministers were expected to be back in London early next month, and that Mr Brown would stage a reshuffle as well as putting together an early mini-Budget.
The past week has seen speculation that there may be one or more challenges to Mr Brown's leadership of the party.
On Tuesday, Labour's deputy leader Harriet Harman insisted she was "not preparing the ground for a leadership election" and said she did not accept Mr Brown's spell as leader was "over".
And the following day Mr Miliband wrote an article in the Guardian which discussed Labour's future without mentioning Mr Brown.
Questions were also raised about Mr Brown's leadership after Labour's shock defeat at the Glasgow East by-election.
Speaking to BBC Radio 4's Today programme on Saturday, Labour MP John McDonnell - who tried to stand against Mr Brown when Tony Blair stood down last year - said those with aspirations to Mr Brown's position should be open about their intentions.
He called for a "democratic election for the leadership", and criticised his colleagues.
"We have got to get away from behind-the-scenes plotting," the MP for Hayes and Harlington said.
And former Conservative party deputy leader Lord Heseltine said Mr Brown was "boxed in" and that the worsening economy meant he could no longer point to his time as chancellor to stifle criticism.
"Trying to reassert authority which depends on old readings of success is not open to him," Lord Heseltine said.