David Cameron has defended his Tory party leadership and warned there will be "no retreat to the comfort zone".
David Cameron faces unease within his own party
It comes as a newspaper report said at least two Tory MPs were calling for a vote of no confidence in him.
Coming third in two by-elections, a row over grammar schools and an apparent "Brown bounce" in the polls have prompted some unease in the party.
Mr Cameron told Sky News the party was back on the "centre ground" with "every chance" of winning the next election.
The Conservatives were beaten into third place by the Liberal Democrats in the two by-elections held in Ealing Southall and Sedgefield last Thursday.
The Sunday Telegraph reported that as many as six MPs had sent letters to the chairman of the party's 1922 backbench committee - although letters from 29 MPs are needed for any vote of no confidence to take place.
Asked whether his Tory critics were going to "get their party back", Mr Cameron told Sky News: "This is the Conservative Party, but what we are not going to do is retreat to the comfort zone.
"I made changes to and with the Conservative Party over the last 18 months for a very clear purpose, to get us back into the centre ground, to get us into a position where people listen to what we were saying, where we are more in touch with Britain as it is today."
He said the Conservatives were ready for a general election "whenever he [Gordon Brown] has the courage to call it", and said they were the party to address Britain's "broken society" - whereas Mr Brown had been at the top of the government that had created it.
"There is going to be a very tough battle for the next election, there's no doubt about it," Mr Cameron said.
"But when I look at what my party has done in terms of getting into the centre, the serious long-term policy work, the team that I have got in place, I think we have every chance of fighting that election, fighting to win and winning it."
In Ealing Southall Mr Cameron's choice of candidate, Tony Lit, proved controversial because he had only recently joined the party, and had been photographed with Tony Blair at a Labour fundraising dinner.
But the Tory leader said Mr Lit had been a "very good candidate" and although the Conservatives had come third, their vote had not been "squeezed" but had held up.
"I'm not satisfied, we should have done better, we need to work hard at it, but now it's on to the next test," he said.
The Sunday Telegraph said at least two, and up to six, Tory MPs were calling for a vote of no confidence in Mr Cameron - but 1922 Committee chairman Sir Michael Spicer has refused to confirm or deny whether he had received any such letters.
The MPs supposedly involved have not been named, and sources close to Mr Cameron have indicated it was not being taken seriously by the party leadership.
Tory frontbencher Andrew Mitchell criticised "gutless and anonymous sniping" and told the BBC it was a "pretty flaky story".
And former Conservative minister Ann Widdecombe, who has said Mr Cameron should do more to shore up the Tories' traditional vote, told the BBC that, if true, the story was incomprehensible.
"I don't understand what they think I going to happen next. I mean, are they really saying - are they really saying - they want another leadership election, when this man has been so consistently ahead? It seems to me an extraordinary thing to do. And it isn't the spirit which wins wars."
Polls for two Sunday newspapers point to a continuing "bounce" effect for Gordon Brown, with the Sunday Times showing a seven-point lead for Labour over the Tories.
The YouGov poll puts Labour on 40%, the Conservatives on 33% and the Lib Dems on 15%.