A technical committee is now searching for a replacement for the head coach, whose team conceded 12 goals in three games and failed to find the net themselves.
The dismissal came late on Sunday, after Brazil's 2-0 win over Germany in the World Cup final. The Germans had beaten Saudi Arabia 8-0 during the competition's group phase.
Before the finals, Al-Johar was considered a saviour, coming to the rescue of the Saudi team after they had initially faltered in the qualifying competition.
Al-Johar earned the respect and trust of the federation as his team scored 47 goals in 14 World Cup qualifiers, making them the top-scoring team of the 32 countries that played in Japan and South Korea.
However, the federation, headed by Prince Sultan bin Fahd, is not known for its patience and perseverance and the disappointing performance over three games last month meant Al-Johar's demise was almost inevitable.