The number of criminals jailed in England and Wales fell by 5% in 2006, according to the Ministry of Justice.
The government has promised to build three new "super-prisons"
The figure dropped from 101,200 in 2005 to 96,000. This continues a trend of year-on-year reductions since 2002.
Fines and community penalties also declined - but there was an increase in out-of-court sanctions, such as fixed penalty notices and cautions.
The latest prison population figures show it has dropped to 81,455 from a record high earlier in November.
Latest statistics also show that the average length of prison sentences fell for people convicted of sexual offences, violence against the person, burglary, robbery, criminal damage and drug offences.
But the BBC's home affairs correspondent Danny Shaw said figures for sentence length do not include offenders given life or indeterminate prison sentences, where a minimum term is set, and so should not be taken as an indication that offenders will serve less time in prison.
On 5 December the government accepted a report by Lord Carter of Coles which proposed a long-term £1.2bn building and modernisation programme.
This includes creating three "super-prisons" by 2014 - each with a 2,500 capacity - to replace older, smaller jails.