Judge Thomas Penfold Jackson said the action was needed to ensure the company - once the world's biggest - did not operate as a monopoly.
He also ordered the firm to modify its conduct to give its competitors a better chance of selling their own software.
Microsoft said it would appeal against the ruling. The implementation of the break-up order is likely to be delayed until the higher courts reach a decision.
The US Justice Department has proposed that the appeal go straight to the Supreme Court - which if accepted would significantly shorten a legal process which has already lasted 10 years.
Windows and the rest
Judge Jackson said the company should be split into two separate businesses:
"Our efforts will protect competition and ensure that consumers will have improved products in the marketplace," US Attorney General Janet Reno said.
The Judge ordered the company to submit a plan for its break-up within four months, with the final division to take place within a year.
He also said that "Microsoft has proved untrustworthy in the past" and gave the company three months to modify its business practices.
Guilty of anti-trust violations
Judge Jackson had already ruled that Microsoft had broken anti-trust laws, abusing its dominant position in the computer operating system market.
The US Government charged, and the court accepted, that Microsoft had forced its customers to adopt its own browser by discounting and putting pressure on computer makers, to the detriment of rival software maker Netscape.
But Microsoft said it encouraged competition, and argued that any break-up would reduce innovation and consumer choice.
"This ruling is unfair ... Consumers every day see lower prices and an economy full of competition," Microsoft founder Bill Gates said.
"Keeping our team together is crucial for our efforts ... we are confident that we will win our appeal," he added.
Conduct remedies
Microsoft's business is likely to be severely restricted by the court's remedies, which are designed to ensure that other companies can compete more effectively with the software giant.
The court ordered Microsoft to modify its behaviour in the following areas:
More legal delays
Despite the government's attempt to take the case directly to the US Supreme Court, the appeals process could still take many years.
The Supreme Court has only agreed to bypass the Appeal Court procedure in one other case - the break-up of huge US telephone company AT&T.
The uncertainty could make it more difficult for Microsoft to recruit and retain its staff.
Microsoft's shares were trading at just over $72, up from its record low of 60, but is still down 40% this year. That has wiped out over $200bn off its stock market value.