Ouaddou was racially abused during Saturday's game against Metz
Referee Damien Ledentu says he was not aware that Morocco and Valenciennes' Abdeslam Ouaddou was being racially abused during a French league game.
Ledentu insists that he would have stopped the match between Valenciennes and Metz if he had been aware of the abuse directed at the Morocco defender.
Ouaddou walked towards the stands at halftime to remonstrate with a Metz spectator who had repeatedly insulted him, although he was restrained by stewards and given a yellow card.
The player lodged an official complaint after the match.
"At no time did he come to me to tell me what was happening," Ledentu said.
"Had I been aware of the situation I would have stopped the match to assess it."
The French league president Frederic Thiriez and the Metz president Carlo Molinari have said they will sue the spectator.
Thiriez said that the strongest possible action would be taken against the individual, so such acts were not repeated.
"In the light of such deviant behaviour, there is only one answer, it is to respond in the strongest manner possible," he told French radio.
"I do not want this type of person in our stadia.
"That means taking legal action and we will be filing a complaint along with the player against the person in question on Monday.
"In the long term it is imperative we do our utmost to ensure that we ban such individuals from the stadia for long passages of time. Such behaviour is intolerable."
Thiriez had admitted before Ledentu's comments that the behaviour of the referee would be looked into.
Metz won the game 2-1.