Government plans to make all passport applicants also have an ID card have been defeated in the Lords.
MPs overturned previous Lords defeats on the ID Cards Bill
Peers voted by a majority of 61 to overturn the proposal - backed by MPs last month - for a second time.
Opposition peers say the plans break the government's promise that ID cards will initially be voluntary.
But ministers say there are no proposals to extend the scheme to holders of other documents. The ID Cards Bill will return to the Commons.
Opposition peers are engaged in a battle of wills with MPs, who have already overturned one Lords defeat on the issue.
The government has not ruled out using the Parliament Act to force through the measure.
Ministers say ID cards should be linked to biometric passports, which are being issued for the first time this week.
Home Office Minister Baroness Scotland of Asthal told the Lords that cards were "in the public interest and in the interests of national security".
They would also help "improve public services", she said.
Peers voted by 227 to 166 against the government to insist that people should be able to opt out of the ID card scheme when they get a new passport.
Conservative shadow minister Baroness Anelay of St Johns called the government's plan "compulsion by stealth".
Liberal Democrat Lord Phillips of Sudbury said: "It's not often it's left to the opposition to make sure the government honours its manifesto pledges."
He said the description of ID card plans as voluntary "stretches the English language to breaking point".
Lord Phillips added: "It's not clear to me what benefits the compulsory scheme will bestow, just as it's never been clear what it will cost."
Conservative and Liberal Democrat peers have dropped their previous insistence that the scheme should not go ahead until the full costs are revealed.
Ministers have said the cards will not be compulsory until there is a further vote of Parliament - and even then people will not have to carry them.
Under the plans, everybody applying for a new passport from 2008 will also be issued with an ID card.
Baroness Scotland said there were no plans to extend this to people applying for other documents.
Biometric passports, also known as e-passports, will be introduced to all applicants by the end of August.
It will include a electronic chip containing measurements of the holder's facial features, such as distances between the nose, eyes and mouth.
The UK and other countries must introduce biometric passports by October to remain part of the US visa waiver scheme, which makes travel to America easier.