Mr Cameron pledged to leave the EPP in his 2005 leadership campaign
Senior Tory William Hague has said the Conservatives' new "anti federalist" bloc in the European Parliament will be "good for European democracy".
The new European Conservatives and Reformists Group includes 55 MEPs from across eight member states.
Former Tory MEP Caroline Jackson warned it would create "bad blood" with traditional centre-right allies.
But Mr Hague said it would still work with the EPP group but differed from it on the extent of European integration.
He told BBC Radio 4's World at One the way the Conservatives had left the European People's Party grouping had been been "perfectly amicable".
However, Foreign Secretary David Miliband said the Tories "have dragged themselves from Euro-scepticism to Euro-extremism".
Mr Hague said: "We do work, on a regular basis, with President Sarkozy's party, with Chancellor Merkel's party - that doesn't mean we have to be in the same group in the European Parliament.
"These groups will often work together but clearly we differ with those parties about the extent of European integration.
"We don't believe in the so-called federal Europe and it has got to be good for European democracy and diversity to have a grouping in the European Parliament with which we agree and can put an alternative point of view."
The Conservatives will be the biggest party in the new group with 26 MEPs, including Northern Ireland's Jim Nicholson of United Conservatives and Unionists - New Force.
Others who have signed up include 15 MEPs from the Poland's Law and Justice Party (PiS), nine from the Czech Civic Democratic Party (ODS), and one each from the Dutch ChristenUnie, Latvian National Independence Movement (TB/LNNK), Hungarian Democratic Forum (MDF) and Belgian Lijst Dedecker (LDD).
One member of Finland's Centre Party will also join the group, although the remainder of the party's MEPs will remain in the liberal ALDE bloc.
Lib Dem foreign affairs spokesman Ed Davey said the Tories had chosen "ideological isolationism" over influence.
He added: "This announcement confirms that the Tories have left the mainstream of European politics and joined forces with a rag-bag of parties with extreme views."
But Mr Hague disagreed. He said the ODS - the party of former Czech PM Mirek Topolanek, who lost a confidence vote in May - were still the "leading party of the Czech Republic".
The PiS were "the party of the president of Poland", he said. Poland is governed by the centre-right Civic Platform.
Other allies in Finland, the Netherlands and Latvia were part of ruling coalitions, he added.
"These are no marginal parties they are mainstream parties that we are very happy to work with," he said.
All members of the new group have signed up to the "Prague Declaration", negotiated in the Czech Republic, which argues for EU reform and opposes federalism.
To form a group in the European Parliament and access EU funding, the Conservatives had needed to attract a minimum of 25 MEPs from at least seven states.
Mr Cameron pledged to cut the Conservatives' ties with the EPP grouping during his 2005 Conservative leadership campaign.
The Tories have also been criticised for joining forces with the PiS, who in the past have tried to ban gay marches in Poland. Mr Hague told the BBC that the PiS's attitudes to gay rights had changed.
Robert Oulds, director of the Eurosceptic Bruges Group, said: "The Conservative Party's federalist rump, both at Westminster and Brussels, should now recognise that the tide has turned."