"At each visit our agents carried bogus badges and identification, declared themselves as armed law enforcement officers and gained entry by avoiding screening. At least one agent always carried a valise," Mr Hast said.
They are being urged to step up security after undercover investigators, carrying fake identification and weapons, gained access to highly-restricted areas.
Posing as law enforcement officers, the investigators by-passed security at 19 federal government buildings, as well as two of the country's busiest airports.
The BBC's Rob Watson in Washington says that the GAO investigation is just the latest in a series of embarrassing security lapses at the very heart of the US government.
'Startling and shocking'
Last year a Russian listening device was found inside the State Department, from where portable computers containing sensitive information have also gone missing.
There are also lingering security concerns after the bombing of the US embassies in Kenya and Tanzania.
Congressmen have described the findings of the investigation as startling and shocking.
The GAO agents used faked credentials obtained from the internet and bypassed metal detectors by saying they were carrying authorised guns.
Our correspondent says the government departments and agencies targeted in the investigation have all been promising to improve security.
'Bogus credentials'
"At no time during the undercover visits were our agents' bogus credentials or badges challenged by anyone," said Robert Hast, GAO Assistant Comptroller General in testimony before a Congressional subcommittee on Thursday.
"At each visit our agents carried bogus badges and identification, declared themselves as armed law enforcement officers and gained entry by avoiding screening. At least one agent always carried a valise," Mr Hast said.
Badges used included a movie prop of a police badge, a counterfeit federal badge and a fake drug task force badge. The investigators used commercially available software to create counterfeit law enforcement identification that bore no resemblance to any genuine credentials, he said.
"At the 21 sites that our undercover agents successfully penetrated they could have carried in weapons, listening devices, explosives, chemical or biological agents," Mr Hast said.
Stepping up security
Pentagon spokesman Bryan Whitman said that, as a result of the GAO investigation, law enforcement officers from other agencies will no longer be given unescorted access to the building, which is the headquarters of the US military.
CIA spokesman Mark Mansfield said the fake officers had only been allowed in the museum and gift shop and never got close to offices of top officials of his agency.
But he acknowledged: "It is a matter of concern they got onto the compound. We take security very seriously. It's a top priority."