Three workers were injured when scaffolding collapsed at a construction site in the centre of Milton Keynes.
One eyewitness said the scaffolding fell "like a pack of cards" from 14 floors. Two of those injured are in a serious condition. The third is stable.
Up to 200 workmen are thought to have been on the site in Witan Gate, but many were on a lunch break when the scaffolding collapsed.
Officials said they thought all those at the site had been accounted for.
All building work has stopped and police are treating the site as a crime scene.
A new Jurys Inn Hotel, due to open in June, and homes are being built at Witan Gate.
Eyewitness Nick Reynolds said: "It came down like a pack of cards." Others said it sounded like a roll of thunder.
The managing director of site contractors McAleer and Rushe, Eamonn Laverty, said the scaffolding had collapsed in high winds
"Our prime concern is for the health and wellbeing of the individuals involved in the incident and to this end we are awaiting an update on their condition from the hospital.
"The company's health and safety officials are currently conducting an internal investigation into the accident."
He said the incident has been reported to the Health and Safety Executive which is also conducting an investigation.
Thames Valley Ch Insp Andy Stanton said the fire brigade had used heat-seeking equipment to search the site but more specialist equipment would be used to comb the wreckage.
"We are working with the ambulance service and Health and Safety Executive to make sure a site search is undertaken and the investigation will be thorough and painstaking."
The three people who were rescued were taken to Milton Keynes General Hospital for treatment.
Two Shires Ambulance Service said: "Apart from the three people taken to hospital a number of others were treated for minor injuries at the scene."
Initially they sent 10 ambulances, two emergency vehicles, two air ambulances, two accident and emergency vehicles and set up two offices.
Radio Five Live listener Christopher Keen said shortly after the collapse: "It looks like absolute devastation. It is a massive building site, there are six tower cranes but the scaffolding is just one pile of matchsticks."
Des Williamson, of Buckinghamshire Fire and Rescue Service, said: "This is as serious as it gets. The initial rescue was reasonably straightforward.
"Since then we have mobilised specialist equipment. We are going to start a very detailed search of the rubble.
"If there are persons trapped under there we will take every precaution and we will find them."