The report said there had been improvement in the last seven years.
Scotland's chief prisons inspector has warned that sex offenders receive the worst preparation before their release.
Andrew McLellan said there was a lack of sex offender treatment programmes in jails - and little evidence they made a difference.
In his annual report, Dr McLellan also said prison overcrowding remained a "real evil".
But he said jail living conditions had been transformed for the better, and paid tribute to prison staff.
Opposition parties said the report showed the Scottish Government was failing to cope with problems in prisons - but Justice Secretary Kenny MacAskill said ministers were taking action to address the concerns.
In his final annual report after seven years in the job, Dr McLellan said that, despite the efforts of the Scottish Prison Service, sex offenders received "the worst preparation for release", despite being the type of prisoner which caused the most public concern.
He said the treatment programme was too small, while the most serious sex offenders who refused to admit their crimes failed to sign up for it.
"Equally disturbing is the fact that there is very little statistical evidence in the UK that the sex offender treatment programme makes a difference," added Dr McLellan.
"It is surely time that research evidence was produced to support a programme which remains the principal resource for addressing the offending behaviour of sex offenders in Scottish prisons."
The chief prisons inspector said many problems were long-standing ones, with no solutions in sight.
He said: "The damage done by overcrowding has already been emphasised - but it cannot be emphasised too often.
"Prisons do not improve more quickly because the best efforts of all the most imaginative and caring staff are completely frustrated by the evil of overcrowding."
Dr McLellan raised concern that under-16s were still being held in prison, and that remand prisoners were still spending too long locked up in cells.
But he also paid tribute to the "patience, humanity and fundamental decency" of all prison staff.
And he said the last seven years had seen a transformation which had produced first class prison buildings - although Dr McLellan said it was "with despondency" he had to report that the practice of slopping out still existed in one jail, Peterhead.
Scottish Conservative justice spokesman Bill Aitken accused ministers of wanting to empty Scotland's jails through the use of community sentences.
"The proper use of prison will eventually lead to a reduction in prisoner numbers, but that will only happen when more police are in our communities, we properly end the scandal of automatic early release, we adopt a zero-tolerance approach to drugs and put in place effective rehabilitation services," he said.
Richard Baker, the Labour justice spokesman said: "Kenny MacAskill's handling of Scottish prisons has been nothing short of disastrous.
'Protecting the public'
"Mr MacAskill's answer to prison overcrowding is to attempt to empty jails by scrapping six-month sentences," he said. "This is a move that will not save the taxpayer a penny but will seriously damage public safety."
The Lib Dems' Robert Brown added: "This report is yet another reminder that the justice secretary hasn't got to grips with overcrowding in our prisons.
"We know that our prisons are getting more dangerous and the rates of re-offending remain high. Prison has an important role to play in our justice system and time spent in prison should be about more than just punishment."
Mr MacAskill said the prison estate was in far better shape than seven years ago, adding that a record £120m was being invested in Scotland's jails.
But he said of Dr McLellan's report: "He is also entirely right to point out that much more needs to be done to deal with overcrowding and other inherited problems such as high reoffending rates.
"This government is committed to protecting the public and prison will always be the right place for serious and dangerous offenders - but modern and proportionate offender management is about more than simply building more and more jails."