Roland Nilsson has left Highfield Road
Chairman Mike McGinnity and chief executive Graham Hover announced the decision on Tuesday following a meeting with the pair.
Nilsson and Smith were told in the meeting that the club's performances this season had been totally unacceptable.
Coventry currently lie eleventh in the First Division, having failed to make the promotion play-offs.
McGinnity said: "We understand the fans' frustration completely, in addition to the need for immediate change.
"The board has therefore decided to terminate the contracts of Roland Nilsson and Jim Smith with immediate effect."
Reserve team manager Trevor Peake and goalkeeping coach Steve Ogrizovic will take charge of the team for the final fixture of the season away at Burnley on Sunday.
Nilsson was appointed as successor to Gordon Strachan in October last year after a spell as caretaker manager in which he guided the club to five wins and a draw in the league.
His unbeaten run extended for a further five Division One games before the Sky Blues lost 3-2 at Millwall at the beginning of November.
Six further defeats in Coventry's next 10 fixtures followed before former Derby boss Smith was appointed as assistant to the Swede on 3 January.
Coventry remained in with a good chance of reaching the play-offs but a run of six games without a win in March and April saw their hopes disappear, and ultimately cost Nilsson his job.
He made his last appearance as a player in the Sky Blues' 4-0 defeat to Preston on 6 April and immediately vowed to hang up his boots.