The four men will not be arrested but will be sent a summons to turn up on 11 March at City of Westminster magistrates court, a short walk from the Houses of Parliament.
Prime Minister Gordon Brown said he was "very angry about what has happened" as he arrived in Exeter for a regional Cabinet meeting.
He said: "We took steps some months ago to remove the right of these people to stand as candidates for the Labour Party. These are very serious criminal allegations. All criminal allegations have got to be investigated. It's a matter now for the courts."
A Conservative Party spokesman said: "Lord Hanningfield has today resigned his position in the House of Lords as shadow business spokesman.
"David Cameron has also asked Lord Strathclyde, leader of the opposition in the House of Lords, to suspend the Conservative whip with immediate effect."
False accounting charges
Announcing the Crown Prosecution Service decision, Mr Starmer said: "In four cases, we have concluded that there is sufficient evidence to bring criminal charges and that it is in the public interest to charge the individuals concerned."
Former minister Elliot Morley, MP for Scunthorpe, will face two charges in relation to a total of £30,000 of mortgage interest claims on a property in Winterton, Lincolnshire between 2004 and 2007.
The charges allege he made claims "in excess of that to which he was entitled" and - for part of the period when "there was no longer a mortgage on that property".
We totally refute any charges that we have committed an offence and we will defend our position robustly
Statement from Elliot Morley, David Chaytor and Jim Devine
David Chaytor, MP for Bury North, is accused of "dishonestly claiming" £1,950 for IT services and further sums of £12, 925 and £5,425 relating to rent claims on properties which he and his mother allegedly owned.
Livingston MP Jim Devine is accused of "dishonestly claiming" money for cleaning services and for stationery using false invoices.
Speaking outside his home he said it was the charges that were being brought were new and could be easily explained, adding: "I'm absolutely distraught and astonished at the decision that's been taken today."
In a joint statement, the three MPs said they were "clearly extremely disappointed" that charges were being brought.
"We totally refute any charges that we have committed an offence and we will defend our position robustly," the statement said.
"We are confident of our position and have been advised by eminent QCs."
They added that the issue should have been resolved by the Parliamentary Commissioner for standards and said while the expenses system had been "utterly discredited" they believed there had been "a complete inconsistency of approach to different individual cases".
Paul White - the Conservative peer Lord Hanningfield - is accused of "dishonestly" submitting claims "for expenses to which he knew he was not entitled" - including overnight stays in London.
CPS statement on MPs' expenses
He denies the charges and says he will "vigorously" defend himself against them adding: "All the claims I have ever made were made in good faith. I have never claimed more in expenses than I have spent in the course of my duties.
"To avoid any embarrassment or distraction for my party, I am standing down from my frontbench duties in the House of Lords with immediate effect."
The Conservatives said he had been suspended from the parliamentary party. He has also quit as leader of Essex County Council.
David Chaytor and Elliot Morley were suspended from the parliamentary Labour party last May.
The solicitors acting for Lord Clarke said he was "hugely relieved that the nine month investigation into his claims under the House of Lords' Allowances Scheme has ended at last, and that he has been cleared of having committed any criminal offence".
Legal expert Joshua Rozenberg explains the charges
Legal analyst Joshua Rozenberg told the BBC that Keir Starmer and the MPs' joint statement implied that the issue of Parliamentary privilege might be used by defence lawyers.
It traditionally guarantees MPs and peers immunity from slander laws for statements in Parliamentary debate and also relates to access to the Parliamentary buildings.
But he said it appeared that lawyers had suggested it may extend to the charges faced - the MPs' statement said it was "an issue that should be resolved by the parliamentary commissioner who is there to enforce any breach of the rules".
In his statement Mr Starmer said: "We have considered that question and concluded that the applicability and extent of any Parliamentary privilege claimed should be tested in court."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.