Mr Koizumi said he accepted that the treatment of the leprosy victims had violated their human rights, and that the government had been guilty of severe prejudice against them.
The apology comes two days after the government decided not to appeal against a court ruling that it must pay compensation to more than 100 former patients.
But in a separate statement, the government disagreed with the court verdict saying it would severely restrict the activities of legislators.
Mr Koizumi said he accepted that the treatment of the leprosy victims had violated their human rights, and that the government had been guilty of severe prejudice against them.
"The government seriously reflects and offers its frank apology over the pain and suffering of the patients and former patients," he said in a statement.
He also announced a series of measures including:
Earlier this month, the Kumamoto district court in southern Japan ordered the government to pay 127 former leprosy patients a total of 1.82bn yen ($14.8m) for failing to change a policy of isolating them after 1960, when drug therapy allowed out-patient treatment.
Inhuman treatment
Under the draconian 1953 Leprosy Prevention Law, which was repealed only in 1996, patients - including young children - were forced to leave their homes and enter the special medical centres.
On Wednesday, shortly before Mr Koizumi announced that his government would not appeal against the Kumamato ruling, a group of former patients met the prime minister.
In an emotional meeting, they described their treatment in the centres and how they were discriminated by friends and even relatives.
"For 60 years, I was not treated as a human," a former patient and plaintiff in the Kumamoto case, Mamoru Kunimoto, said.
"But the fact that the government will not appeal has given me back my humanity."
Mr Koizumi said the decision not to appeal was made on humanitarian grounds but maintained that he did not agree with the court ruling.
Some members of the government are worried about the impact of the ruling which, they fear, could set a precedence for other compensation claims.