Conservative leader David Cameron says there are no longer any safe seats in Wales for Labour after his party topped the poll in the European election.
Mr Cameron travelled to Cardiff to celebrate the results with party activists after Labour was forced into second place in Wales.
Labour, Conservatives and Plaid Cymru won a seat each, as did the United Kingdom Independence Party.
Welsh Labour politicians said the election was a "wake-up call".
Speaking outside the Senedd in Cardiff Bay flanked by Ms Swinburne and Nick Bourne, who leads the Tories in the Welsh assembly, Mr Cameron said it was an "historic day" for the party in Wales.
"Not since the time of Lloyd George has a party other than Labour won in Wales and today it is the Conservative party that has done that in Wales," he said.
"No longer should people say in politics that there are uncrackable bastions, safe seats where Conservatives cannot go.
2009 EURO ELECTION - WALES
Conservative: 145,193 (21.2%, up 1.8%)
Labour: 138,852 (20.3%, down 12.2%)
Plaid Cymru: 126,702 (18.5%, up 1.1%)
UKIP: 87,585 (12.8%, up 2.3%)
Lib Dems: 73,082 (10.7%, up 0.2%)
Greens: 38,160 (5.6%, up 2%)
BNP: 37,114 (5.4%, up 2.5%)
Christian Party: 13,037 (1.9%)
Socialist Labour Party: 12,402 (1.8%)
No2EU Yes to Democracy: 8,600 (1.3%)
Jury Team: 3,793 (0.6%)
Turnout: 30.4% (down 11%)
"And it is no longer the case that people can say that the Conservative party cannot win in Wales, we have won here in Wales."
Wales' MEPs are Conservative Kay Swinburne, Labour's Derek Vaughan, Plaid Cymru's Jill Evans, and UKIP's John Bufton.
The turnout of 30.4% was down by 11% on the previous European elections in 2004.
In 2004 Labour had won two of the four seats with 32% of the vote. But five years on Labour has seen its vote fall to just 20%.
The Tories edged ahead of Labour, and were up 1.8% to 21%, while Plaid improved its share by 1.1% to 19%.
But Liberal Democrats again suffered European disappointment. They have never had a Welsh MEP, and were pushed into fifth place, with 10.7%, up 0.2%.
It is the first time since 1918 Labour has failed to come first in a Welsh election, as its vote dropped by 12%.
Conservative Cheryl Gillan: People in Wales are thoroughly fed up with the Labour Party
Mr Bourne said: "This result kills the lie that Wales will always be Labour."
Speaking on BBC Radio Cymru, former Labour MEP Eluned Morgan, who retired at the election, said the message from Labour to voters was not clear.
"There was a problem with our message. I was part of our campaign and I can't tell you clearly what our message was," she said.
"This was a kicking and we have to rebuild the party in Wales. That won't be easy.
"There's a myth about a Welsh Labour machine. There isn't much of a machine and at the moment lack of money really worries us as a party."
But Welsh Secretary Peter Hain said Labour could recover and added that huge numbers of Labour voters had stayed at home because of a "toxic" combination of the recession and the MPs' expenses scandal.
"I think it sends a very clear message to everybody in Wales - if you don't vote Labour, if for example you vote Plaid Cymru, you will get the Tories," he said.
"We need to change as a party in Wales because this is a really serious result."
Julie Morgan, Labour MP for Cardiff North, said she supported Gordon Brown as the party's leader and that the people she had spoken to while campaigning did not blame him for many of the issues they were facing.
WELSH PARTIES' PERFORMANCES IN PAST EUROPEAN ELECTIONS
"I believe we have to sort our policies, we have to think why it is that Labour is not appealing to people and to go past starting a leadership election," she said.
"I think that would be a panic response and would not benefit the party or the country.
"We have to look at why large numbers of people in Wales have not voted Labour."
Vaughan Roderick, BBC Wales' Welsh affairs editor, said the Conservative party topping the poll was a "historic achievement" but its share of the vote had only risen slightly on the 2004 elections because it was split among many parties.
"But it was still enough to beat Labour," he said.
Conservatives top the European election poll in Wales, and UKIP has its first Welsh MEP
"As we dig through these figures there are some extraordinary results showing Labour's weakness.
"In Montgomeryshire, which is a Liberal Democrat seat, Labour came sixth. They came behind the Conservatives, UKIP, the Lib Dems, Plaid Cymru and the Greens and they were only just ahead of the BNP."
Plaid Cymru MP Adam Price referred to an earthquake which hit south Wales on Friday night, adding: "We've had an even bigger one today. It's off the Richter scale."
The Carmarthen East and Dinefwr MP said: "The political landscape has changed here. We've benefitted; we've had a good result, the Tories have had a very good result, but Labour... this is earth shattering for them".
He said Labour was "imploding in the hardest of its heartlands".
In 2004 Welsh Labour won 32.5% of the vote, Conservatives 19.4%, Plaid Cymru 17.4%, UKIP and the Liberal Democrats 10.5% each, Greens 3.6%, and BNP 3%. There was a turnout of 41.9%.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.