He led the U's to eighth place in League One this year after succeeding Paul Lambert.
Coleman lost his job after a late-season collapse for the second season running. The Sky Blues have finished 21st, 17th and 19th in the previous three seasons.
Coventry chairman Ray Ranson said: "He is considered to be one of the brightest young managers in the English game and we believe he can take us to the next level.
"We look forward to working with him in the coming weeks, months and years as we bid to get Coventry City back where it belongs."
Ranson's Colchester counterpart Robbie Cowling said: "I'm obviously disappointed to lose a good manager who could have took the club on further than we have done and I thought we started something we've not quite finished," he said.
"Obviously I understand some of the reasons why he's wanted to take up the job at Coventry. They are a bigger club we have to accept that.
"They are in a higher division and it is just 30 minutes away from where he lives with his young family so I think its understandable.
"It's a mixed reaction, I'm very disappointed because we've lost a good manager but I'm also really confident that as a club we are going to go out and find another really good manager, that will take us on."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.