Former Communities Minister Shahid Malik has lost his seat in Dewsbury to the Conservatives.
Mr Malik, who was elected to the constituency in 2005, received 17,372 votes, while Tory Simon Reevell gained the seat with 18,898 votes.
Liberal Democrat Andrew Hutchinson came third with 9,150 votes.
Labour's Ed Balls narrowly won the new seat of Morley and Outwood with 18,365 votes, followed by Conservative Antony Calvert who gained 17,264 votes.
The children's secretary won with a 2.3% majority - 7,568 votes less than he received in 2005.
James Monaghan, for the Lib Dems, came third with 8,186 votes.
Mr Malik said boundary changes incorporating traditional Conservative areas meant retaining the Dewsbury seat was always going to be an "uphill struggle".
"As I accepted the will of the people five years ago, I accept it today," he added.
Mr Reevell said: "The voters have decided enough is enough."
Mr Malik had been Dewbsury's MP since 2005
In other constituencies, the Conservatives have taken the Colne Valley seat from Labour, with Jason McCartney gaining 20,440 votes.
The Tories have also taken the Pudsey seat from Labour with Stuart Andrew receiving 18,874 votes - 1,659 more than Labour's Jamie Hanley.
Labour's Gerry Sutcliffe retained his Bradford South constituency, receiving 15,682 votes, while Conservative Kris Hopkins took the target seat of Keighley from Labour's Jane Thomas, with 20,003 votes.
Labour's Jon Trickett retained the Hemsworth constituency with 20,506 votes and the Liberal Democrats held on to Leeds North West constituency, with 20,653 votes for Greg Mulholland.
The Shipley seat was held by the Conservatives with Philip Davies receiving 24,002 votes.
The Normanton, Pontefract and Castleford constituency was retained by Labour's Yvette Cooper who received 22,293 votes.
The seat saw a swing of 12.5% from Labour towards the Conservatives.
Meanwhile the Conservatives took the Elmet and Rothwell constituency - one of its target seats - from Labour with candidate Alec Shelbrooke receiving 23,778 of votes.
The marginal seat of Calder Valley also saw a Conservative gain from Labour, with Craig Whittaker polling 20,397 votes.
Bradford West was held by Labour and the party also retained its Halifax seat where Linda Riordan was re-elected with 16,278 votes.
Liberal Democrat David Ward took the new Bradford East seat from Labour's Terry Rooney, with a swing of 7.6%. It was a closely-fought contest, with Mr Ward polling 13,637 votes and Mr Rooney receiving 13,272.
Labour has retained control of Leeds East, Leeds West, Leeds North East, Leeds Central, Batley and Spen, Huddersfield and Wakefield.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.