The Tories have told Chancellor Alistair Darling to "get a grip" after he admitted the details of all UK child benefit records were lost in the post.
Mr Osborne said millions will worry about their bank account security.
Shadow chancellor George Osborne said the HM Revenue and Customs error, which includes child benefit and bank details of 25m people, was "catastrophic".
"This is the third and most serious breach at HMRC," said Mr Osborne.
He accused Mr Darling of lurching "from one crisis to another" and of failing every family in the land.
He said the problems should mark the "final blow" for the government's identity card programme.
"They simply cannot be trusted with people's personal information," he said.
He spoke out after the chancellor revealed the child benefit details, contained on two discs, had been sent by HMRC to the National Audit Office, but never arrived.
"The prime minister says that the first duty of government is the protection of the citizen - and today the government is responsible for breaching that duty of protection to 25 million people," he said.
"Half the country will be very anxious about the safety of their family and the security of their bank accounts and will be wondering how on earth the government allowed this to happen."
Mr Osborne questioned why it had taken the chancellor four days to contact the police after finding out about the "fiasco".
He asked when the prime minister was told - to which Mr Darling replied: "Within half an hour of me being told."
And he asked what the point was of the Commons passing laws to protect the privacy of personal information "if those laws aren't enforced at the heart of government".
In a personal attack on Mr Darling, he declared: "Since you came to office less than six months ago you have lurched from one crisis to another.
"Now your department has compromised the security and safety of every family in the land.
"This autumn the prime minister said he had shown this government could be competent and how he needed to set out his vision.
"There are 25 million people whose personal details have been lost by this government.
"Never mind the lack of vision just get a grip and deliver a basic level of competence."
Mr Darling said the problem was "inexcusable". "HMRC has well laid down procedures and there was no excuse for breaching those," he said.
"It's a matter of extreme regret that so many people are going to be caused anxiety."
The chancellor defended the government's plans to introduce ID cards. He said that without the protection of the scheme, information was more vulnerable than it should be.
"This is a deeply regrettable incident - it should never have happened," he added.