Justice Minister Cathy Jamieson has defended the Scottish Executive's record of prison reform following an attack from an outgoing prison chief.
The retiring governor of Inverness jail, Alastair MacDonald, told BBC Scotland that the prison service had suffered years of neglect.
Mr MacDonald also raised concerns that prisons were being turned into a social service for the vulnerable.
Ms Jamieson admitted the need to address reoffending rates.
However, she said major investment in infrastructure was improving the prison service across Scotland.
Ms Jamieson said: "We are trying to tackle the reoffending for the first time ever.
"We have actually had a focus on this. We have changed the legislation, we have put in place new ways to do that.
"We are also bringing in measures through legislation to, for example, deal with fine defaulters in an entirely different way so that they don't end up in prison at the public expense.
"The public expects offenders to be punished but also rehabilitated. We want to do that."
The Scottish Prison Service (SPS) also defended the executive's record of investment.
Mr MacDonald said overcrowding meant reverting to practices of 200 years ago with prisoners being sent to jails far from their families.
He also said 30% of those coming through the gates could not read or write and half of them had not reached primary six academic level.
Mr MacDonald, who retires this week after 31 years, said prisons were increasingly becoming a social service.
He said the population at Inverness, also known as Porterfield, had grown in the six-and-a-half years he has been in charge.
"When I first arrived here the daily population was around 125 and now we are regularly having 150," said Mr MacDonald.
"On top of that, on a weekly basis we have to send prisoners down south and indeed this takes us back to the situation that prevailed in the Highlands 200 years ago.
"So we are going in a reverse direction at the moment sadly."
Mr MacDonald added: "It is a very sad fact so many families now have to travel great distances to see their brothers, husbands, whoever, because we have to send them to places like Barlinnie because of overcrowding."
The former governor of Shotts and Barlinnie ruled out any attempt to close Inverness, but said there was a "lack of will at many levels" to deal with unsuitable prisons.
Citing statistics from two years ago showing that 135 per 100,000 people in Scotland were imprisoned, Mr MacDonald said: "A serious question has to be asked of the Scottish nation.
"Why are we having to lock up so many of our citizens?
"What has gone wrong? Why is the direction taking a wrong turning with regard to people who cause risks within our communities."
He said more appropriate and less costly alternatives to prison had to be found for some of those who end up in jail.
"If you look at the average prison population now more than 30% of prisoners coming through Inverness' gates cannot read or write," said Mr MacDonald.
Prison officials pointed to record investment in the service
"About 50% of them haven't reached primary six academic level."
He said 70% of admissions at Porterfield had substance abuse problems and 70% displayed psychiatric illness.
"Many of my senior colleagues now are becoming acutely aware that we are becoming centres of social service," he said.
"We are lifting many people out of poverty so it's claimed. We are certainly lifting many people into prison."
The governor said the Scottish Executive, in consultation with the prison service, had produced helpful and valuable policies.
However, he said they were not backed by adequate funding.
"Many policies are like the Easter eggs we hated getting as children - very large, very attractive but when you take off the silver paper it is a big hollow egg," said Mr MacDonald.
The Scottish Prison Service said the executive had made record levels of investment in renewing the prison estate over the last five years and was currently investing £1.5m a week.
A spokesman added that the issue of alternatives to jail had been recognised by Scottish ministers.
Scotland, he argued, had a great range of alternatives to custody and there had been a significant improvement in facilities in the community.
The spokesman said importance should be placed on the quality of prison services and facilities - not whether or not they were resourced from the public or private sector.
Scottish Conservative justice spokeswoman Margaret Mitchell said Scotland should be following Ireland and Spain.
She said: "Spain imprisons four times more people than Scotland relative to recorded crime and Ireland three times more.
"Unsurprisingly, the deterrent is such that crimes per capita in both Ireland and Spain are around a quarter of the level here."