The Chinese and South Korean governments have summoned the Japanese ambassadors to lodge formal complaints about Mr Koizumi's visit to the Yasukuni shrine.
In the South Korean capital, Seoul, hundreds of people took part in demonstrations for the second day running. And in Taiwan, the government said Mr Koizumi's visit hurt the feelings of Japan's neighbours.
The Yasukuni shrine commemorates all Japan's war dead, including executed war criminals, and many neighbouring countries view it as a symbol of Japanese wartime aggression.
Despite the anger abroad, several more senior Japanese ministers, including a minister in charge of financial affairs, Hakuo Yanagisawa, visited the shrine on Tuesday.
More have said they will attend on Wednesday, the anniversary of the Japanese surrender at the end of World War II. Mr Koizumi moved his visit forward two days in response to pressure not to attend on this highly symbolic date.
Protests
Demonstrators registered their protests across the region.
In the Philippines, 30 elderly women, who said they were forced to work as sex slaves for Japanese soldiers in World War II, gathered outside its embassy in Manila.
"We want compensation! We want it now!" they chanted.
In Beijing, a lone demonstrator tried to unfurl a banner in front of the Japanese embassy but he was stopped by police.
And in Hong Kong, a small group of demonstrators burnt Japan's imperial flag and a picture of Mr Koizumi outside the country's consulate.
They accused Japan's prime minister of celebrating militarism and shouted: "Paying tribute to war criminals is shameful!"
It is the first time since Yasuhiro Nakasone attended on the 15 August anniversary in 1985 that a serving Japanese prime minister has made an official visit to the Yasukuni shrine.
In July 1996, Prime Minister Ryutaro Hashimoto's unofficial visit on his birthday drew so much international criticism he cancelled subsequent visits while in office.