The cartel practice involved the use of false invoices.
Construction giants Balfour Beatty and Carillion are among those the OFT accuses of taking part.
"Cartel activity of the type alleged today harms the economy by distorting competition and keeping prices artificially high," said OFT chief executive John Fingleton.
Office of Fair Trading explains how the 'rigging' worked
"This investigation, together with the OFT's previous decisions in the roofing sector, will hopefully send out a strong message to the construction industry about the seriousness with which we view suspected anti-competitive behaviour.
"Businesses have no excuse for not knowing and abiding by the law."
The OFT said its investigations first started in 2004 following an initial complaint regarding building contracts at Nottingham's Queens Medical Centre.
And on public projects, it was council tax-payers who were among the victims of the collusion, said Edward Walsh of the Local Government Association.
"We'll be seeking to find out whether or not councils will get any of this money back in the shape of fines," Mr Walsh said.
"Ultimately, council tax payers have had to pick up the bill if collusion has taken place."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.