Gordon Brown and opposition leaders welcomed the appointment of Mr Bercow, who stood on a platform of reform and pledged to heal public "anger".
But it is believed some Conservative MPs hope to be able to mount a challenge to unseat him after the next election.
In an interview with the BBC, Speaker Bercow defended his own expense claims, insisting that he had paid money back voluntarily to cover the tax he had saved after he was accused of "flipping" his home. He said he had "behaved honourably".
Asked whether he would reverse the decision to black out MPs expense claims, he said that he favoured greater transparency but would need to consult on how information was presented in future.
Mr Bercow said he would ditch the job's traditional tights and wear a business suit and "smart but unfussy" gown for his role, adding: "The wig has gone."
He added that he wanted to see "brisker progress" in the day to day business of the House of Commons with shorter questions and answers and a "more considered" approach to business.
Questioned whether he was prepared to discipline the prime minister and the leader of the opposition to achieve this, he answered: "Quite."
What they regret is the partisan way he has been elected, with some Labour MPs backing Bercow, not because they thought he was the right reforming Speaker, but because they knew it would anger the Tory benches
Mr Bercow added that he was keen to move away from the "boorishness" which he said had "disfigured" the Commons.
The Speaker acknowledged that many in his own party were opposed to his own candidacy, but suggested that this was because they were used to someone of a different generation holding the position.
He said that he would win their backing "by good performance".
Mr Bercow's first challenge will be to establish his authority over the House.
Mr Bercow is unpopular with large swathes of the Conservative Party after undergoing a political journey from right-wing Thatcherite to outspoken social liberal rumoured to have been close to defecting to Labour.
He won the support of many Labour MPs, who are convinced he will stand up to Conservative leader David Cameron, but there was a marked lack of applause from the Tory benches.
Sir George Young, the rival candidate he defeated in the last round of voting, urged his Conservative colleagues to get behind the new Speaker.
Asked about reports some Tories planned to oust him after the next general election, Sir George said: "I think it would be wrong for an elected party of whatever complexion to use its majority to unseat a Speaker, the moment you do that you have politicised the Speakership."
New Speaker John Bercow chairs his first question time
He added: "Give him a chance, we had an election, we had a fair contest, he won and let's give him the support I think he is entitled to."
Mr Bercow's election was given royal approval in a traditional Parliamentary ceremony on Monday night.
He replaces Michael Martin who quit after nine years amid the furore over MPs' expenses.
He was welcomed by party leaders but faced a mixed reception from Conservative MPs.
One of them, Nadine Dorries, told BBC Radio 4's Today programme that only three Tories had voted for Mr Bercow, and said his election had been "made through a red mist" as "the last hurrah of a dying Labour government".
Alan Duncan, the shadow leader of the Commons, said it would be "churlish" to take anything away from his victory and said Mr Bercow deserved the respect of the House following his election.
But he said: "A lot of Conservatives feel that John positioned himself in order to woo Labour to get the Speakership. A lot of people are annoyed that it worked."
Constitutional expert Professor Vernon Bogdanor of Oxford University said: "We saw him being dragged to the Speaker's chair yesterday, but he's now got to drag the House of Commons in modern times - that's how he'll be evaluated."
ROUND-UP OF OPINION ON NEW SPEAKER
As John Bercow starts work as Speaker of the House of Commons, columnists, commentators bloggers and analysts discuss what this means for politics in the UK.
On the Today Programme, Conservative MP Nadine Dorries says John Bercow won for tactical reasons and MPs aren't taking reform seriously:
Now that he's been elected I hope he can do the job. But unfortunately I think this is more about last night a sort of last hurrah of a dying Labour government and I think it was an appointment that was made through a red mist. It was almost a two-fingered salute to the British people from the labour party and to the Conservative Party because the Labour Party voting for John Bercow had nothing to do with reform.
All the candidates yesterday portrayed themselves as reformers. But this muddied rather than clarified what the Speaker can do in practice The Speaker has no role at all on more ambitious ideas for recalling MPs who have been found guilty of financial abuse or for holding primary elections to select candidates. These are matters for Parliament as a whole and for the parties.
What is clear is that the contest for Speaker has been a dramatic but ultimately unedifying affair involving tactics (not Mr Bercow's) more suggestive of greyhound-nobbling than the installation of the First Commoner of the Land. Yet the problem lay less in the failings of the candidates than in the false hope invested in the contest.
He has to demonstrate his "complete impartiality", if he is to avoid being ejected by the incoming Tories next year. There will also be the traditional tests to his authority in the House Equally though, Bercow will change, as people do when they achieve high office. And sometimes they change for the better (excuse the note of surprise in my voice). That's something to look forward to.
The election of a new Speaker was meant to mark the start of the great clean-up of Westminster. It offered MPs a chance to set party politics aside and work together towards regaining the public's trust. Instead, the Labour majority showed themselves at their most contemptible, swinging behind Tory left-winger John Bercow for no better reason than that he is bitterly disliked by his own party.
A Speaker has three roles: overseeing the complex management of Westminster and its staff; being its public face abroad (the domestic role must now grow); and controlling the Commons, MPs and ministers alike. The last now matters most.
They may have good reason to loathe Mr Bercow for turning his back on them - but their behaviour yesterday was embarrassing as well as childish. It was a poor advert for the Commons which needs all the help it can get in improving its standards.
There is no point undermining him before we see how he actually performs in the job. There is now an opportunity for a great reforming Speaker to drive through reforms that regain for Parliament the ability to hold the government to account and in check.
On Wednesday he should ensure that the Prime Minister answers questions rather than asks them. The first time a government minister announces a policy on the Today Programme, he should be hauled before the Commons to explain himself. Although he must be given a little time to find his feet, he also needs to send a clear signal that the House is now under new management.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.