Computer test was deemed 'risky'

Many pupils were involved in piloting the new test

A computer exam intended to be taken by all England's 14-year-olds would have involved "a substantial risk", an evaluation by the exams watchdog said.

The Qualifications and Curriculum Authority blames itself and the test's developer, but also cites "tensions" in the government's original remit.

Its report questions the reliability of the results pupils were obtaining.

Ministers have already accepted the QCA's advice not to make the Key Stage 3 test compulsory from 2008.

Most pupils found the pilot version difficult and half could not complete it in the time available, the QCA said.

Reliability issue

The computer-based test involved a series of tasks such as e-mailing, spreadsheet work and data handling.

It was innovative and did not use commercial software such as Microsoft's Office suite - which pupils would have preferred.

The QCA's internal evaluation, by electronic assessment researcher Andrew Boyle, defends the decision to use non-standard software.

The prime contractor was Research Machines (RM), which provides ICT software and services to a host of UK educational institutions.

But concerns included the reliability of results when pupils were tested then re-tested, and test results were out of kilter with their teachers' assessments of their ICT abilities.

In 6% of cases their results were two whole test levels below their teachers' assessments.

'Comfort zone'

"At the time of writing it is not possible to make a completely watertight argument that the KS3 ICT test provides reliable measurement," Mr Boyle wrote.

It would have been "a risky test in a risk-averse environment".

He stresses that responsibility for the "state" of the test model lies within the project - with the development agency, the QCA and its National Assessment Agency, and others who approved work or provided advice.

But he says "the obligation to develop an innovative e-assessment implicitly required any developer ... to go outside the comfort zone offered by traditional techniques".

There was a "tension" between the two demands of the Department for Education and Skills.

It had demanded that the test be robust enough to be used in "high stakes" national testing but also an innovative demonstration of ICT-based assessment.

Part of the evaluation involved getting the views of those involved, including 434 pupils in 16 of the schools that conducted pilot tests.

The report says 42% did not understand the instructions, 52% did not have enough time to complete the test and 69% found it difficult or very difficult.

Pupils said they liked using a computer for the test, e-mailing and learning and experimenting with the software.

But many, asked what they had liked about it, replied "nothing" or "finishing it".