Gordon Brown has ordered security checks on all government departments to ensure data is properly protected after the loss of 25m child benefit records.
Under the plans, the Information Commissioner will be given powers to carry out spot checks - a move previously rejected by ministers.
In the Commons, the PM apologised for the "inconvenience and worries" caused.
Conservative leader David Cameron said the government had "failed in its first duty to protect the public".
The Revenue and Customs data on the two missing discs includes names, dates of birth, bank and address details.
During a heated prime minister's questions session, Mr Brown told MPs: "I profoundly regret and apologise for the inconvenience and worries that have been caused to millions of families who receive child benefits.
"When mistakes happen in enforcing procedures, we have a duty to do everything we can to protect the public."
He said he had ordered a review by the Cabinet Secretary of data safety in government and would give Information Commissioner Richard Thomas the power to spot check departments to ensure data is safe.
MISSING DATA INCLUDES
National insurance number
Name, address and birth date
Names, sex and age of children
Bank/savings account details
He also said that "the idea that we are complacent about this is quite ridiculous. We are taking all the action that is necessary."
But Mr Cameron said people would "be angry that the government has failed in its first duty to protect the public".
He added: "What people want from their prime minister on a day like this is to show some broad shoulders, be the big man and accept some responsibility."
Mr Cameron said people were "desperately worried" and they would "find it frankly weird" that Mr Brown still wanted to go ahead with plans for a national ID cards scheme and register.
Mr Thomas said it was "almost certain" that HMRC had broken the Data Protection Act. He told the BBC: "It is a shocking case. I am at a loss to find out what happened in this situation."
He welcomed Mr Brown's announcement that his staff will be able to spot check government departments, adding that it was a power he had wanted for "some time". He also called for "security breaches of this magnitude" to be made "a criminal offence".
The entire child benefit database was sent via internal mail by a junior official from HMRC in Washington, Tyne and Wear, to the National Audit Office (NAO) in London via courier TNT on 18 October.
CHILD BENEFIT HOAXES
Hours after the blunder emerged, a website claimed to be offering the missing data for download
Fullreleases.com, a site registered in Arizona, lists several files entitled "child benefit", but users must pay $29.95 plus tax to join before they can view them
However, once money is paid the files cannot be accessed and the site appears to be a scam
The HMRC said people should take care with any transaction on the internet and anyone concerned about their data should call the helpline
Chancellor Alistair Darling said the civil servant had broken the rules by downloading the data to disc and sending it by unrecorded delivery.
Edward Leigh, the Conservative chairman of the Commons public accounts committee, said the NAO had only asked for basic details about child benefit recipients, without information on personal bank accounts, but was told by "high level" officials that it would be "too burdensome" for HMRC officials to separate out this data.
He said he had been given a copy of a briefing note written by NAO head Sir John Bourn for the chancellor, which suggested that senior HMRC officials authorised the release of sensitive information.
The note says the NAO requested data on child benefit claimants in a "desensitised" form, with bank accounts and other personal data removed, in March, said Mr Leigh.
An email from a senior business manager at HMRC stated that the data would not be desensitised, according to the auditor general's note.
Mr Leigh said the reason given for turning down the NAO request was that desensitising information would require an extra payment to data services provider EDS.
Bosses at the Revenue were not told about the loss of the discs until 8 November, and Mr Darling and Mr Brown learned about the situation on 10 November.
The officials involved waited before informing their superiors in the hope that the discs would be found.
One of the junior officials involved is reported by the London Evening Standard to be in "hiding" in a hotel, with representatives from HMRC.
The Metropolitan Police is leading the search for the discs, and the Independent Police Complaints Commission, which oversees the HMRC, is investigating the security breach.
Her Majesty's Revenue and Customs has set up a Child Benefit Helpline on 0845 302 1444 for customers who want more details.
LOST CDS - SEQUENCE OF EVENTS
18 October - Junior official from HMRC in Washington, Tyne and Wear, sends two CDs containing password-protected records to audit office in London through courier TNT, neither recorded nor registered
24 October - When package fails to arrive, second one is sent by registered post and arrives safely
3 November - Senior managers are told first package has been lost
10 November - Prime minister and other ministers are informed
12 November - HMRC tell ministers CDs will probably be found
14 November - When HMRC searches fail, Metropolitan Police are called in
15 November- Richard Thomas, Information Commissioner, says remedial action must be taken before public is informed
20 November - HMRC Chairman Paul Gray resigns; Chancellor Alistair Darling makes announcement to House of Commons