The memory stick was found outside a pub in Cannock, Staffordshire
An inquiry has been launched after a memory stick with user names and passwords for a key government computer system was found in a pub car park.
Subcontractor Atos Origin, which lost the stick, said there had been a "direct breach" of its procedures.
It said the matter was being taken "extremely seriously" but the integrity of the website, which was closed for a short time, had not been compromised.
Prime Minister Gordon Brown said the company would have "to explain itself".
The data breach is the latest in a series of embarrassing blunders regarding government information.
The pub car park where the memory stick was found
Work and Pensions Secretary James Purnell issued an apology on Saturday after he left confidential ministerial correspondence on a train.
Mr Brown said Cabinet Secretary Gus O'Donnell was sending out fresh instructions to ministers over how sensitive data must be handled.
"There are very strict rules about information being outside buildings and these have to be followed," he said.
"This recent case with a private company, where information about individuals has been lost, makes me even more determined that we will root out this problem about leaving things around."
'No conceivable risk'
The Gateway website allows members of the public to access hundreds of government services including self-assessment tax returns, pension entitlements and child benefits.
The Mail on Sunday, which broke the story, said the memory stick might allow someone to access the personal details of the 12 million people registered on it.
Gordon Brown says he is determined to root out the problem
But a spokeswoman for the Department of Work and Pensions said the device contained user names and passwords for testing an old version of the system, and all the information was encrypted.
"We are taking this issue extremely seriously and a full and urgent investigation is under way," she said.
"We have moved immediately to make sure there is no conceivable risk to users of the Government Gateway, and are convinced the integrity of the Government Gateway has not been compromised.
"On the basis of an initial examination of the contents of the memory stick, it is our experts' opinion that the contents would not allow anyone to breach the very strong security safeguards protecting the website."
It is clear that the employee removed the device from company premises in direct breach of our own operating procedure
She said the system had been shut down "for a short period as a precaution". It re-opened later on Sunday afternoon.
A spokesman for Atos Origin, the government subcontractor which runs Gateway, said one of its employees had "misplaced" the stick.
It was found in the car park of the Orbital Pub in Cannock, Staffordshire, near to where the firm is based.
"The company takes the loss of this device very seriously and we are currently carrying out a full investigation of both the circumstances surrounding its loss and the data content of the stick," she said.
"It is clear that the employee removed the device from company premises in direct breach of our own operating procedure.
"Atos Origin is working very closely with the government and the police. The company takes full responsibility for this loss and will discipline the individual involved.
"As this may become a criminal matter for the individual concerned, it is inappropriate for us to comment further at this stage."
This is the latest in a series of incidents in which memory sticks and other devices holding personal data have been lost.
In August, another government contractor PA Consulting lost a memory stick containing the personal details of tens of thousands of criminals.
Following the latest incident, the Lib Dems said there should be a ban on memory sticks being used to carry such sensitive information.
"Why should we trust the government for its database or ID cards system when they simply cannot be trusted with information," said the party's transport spokesman Norman Baker.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.