Luis Aragones said his family asked him to quit as Spain's coach because of the reaction in the English press over remarks about Thierry Henry.
Aragones' controversial remarks were followed by abuse of England's black players in November's international.
"I have been insulted by being called racist and nobody had called me that before," Aragones told Spanish radio.
"I have suffered a lot in the past few days. I never thought about resigning but my family have asked me to leave."
Aragones has refused to condemn the insults aimed at England's black players after the friendly on 17 November, and remained unrepentant over his comments on Henry.
"You are a slave to your words but, when I said what I said, I was in a private environment," he said.
"Then in places such as the English press, they have insulted me by calling me a racist.
"I think it is wrong that they have been saying Spain is racist and that I am too."
The 66-year-old has revealed he had received the support of Angel Maria Villar, who was re-elected as president of the Spanish FA (RFEF) last Friday.
The RFEF still have to decide on a punishment for Aragones, following recommendations from Spain's Anti-Violence Commission, but it appears that whatever happens, they will not ask him to resign.
"I put my position in Villar's hands, although he was the same person who hired me.
"If someone else had won the elections, I would have done the same thing, although I have heard they also counted on me," he said.
RFEF president Angel Maria Villar will appear before the Anti-Violence Commission on 10 December.
Villar will attempt to explain the comments made by Aragones in the build-up to Spain's friendly with England last month.