Figures published by the Scottish Executive show second year pupils fell below targets in the three Rs - reading, writing and arithmetic.
While children perform well in primary education, many struggle to master basic skills in secondary school.
Opposition parties described the figures as "a damning indictment" of the education system.
The executive's statistics for the year from June 2001 show:
Education Minister Cathy Jamieson said standards must improve, but welcomed a slight improvement in attainment at all levels.
She said: "Quite simply that gap is unacceptable and must be closed.
"Every school needs to pay more attention to ensuring that programmes for reading, writing and mathematics to develop pupils' skills, build effectively on previous learning and ensure that every young person's learning needs are recognised and catered for."
It was vital to get the basic skills right to make progress, the minister added.
'Blighted' lives
Scottish National Party education spokesman Mike Russell said the executive's pledge to improve education standards had been a "hollow promise".
He said: "The tragedy of Labour's failure is that thousands of children's lives are going to be blighted by the executive's inability to address low standards in schools.
"Their approach is to train children to pass exams. Our approach will be to teach them the core skills they will need for life."
The Conservatives said any performance improvement was down to policies introduced when they were still in office.
Brian Monteith, Tory education spokesman, said: "Let there be no doubt. The improvements announced today are the fruit of the radical education reforms of the last Conservative Government.
"After originally opposing these policies, Labour and the Lib Dems now try to take credit for them, and the SNP would still abandon them."
The National Survey of 5-14 Attainment Levels is conducted each year by the Education and Children Statistics Unit of the Scottish Executive's Education Department (SEED).