|You are in: Education|
Friday, 27 September, 2002, 19:13 GMT 20:13 UK
The 'accident waiting to happen'
Mike Tomlinson's key recommendation is that students' marked work in some A2 units in some subjects be re-graded and their overall A-level results be adjusted if necessary.
He needs more information on this from the exam boards, which should be available by next Tuesday.
He told journalists he expected the re-grading could be done quickly after that.
It would apply "where changes to grade boundaries appear to have been out of line with historical patterns" for the old, so-called "legacy" A-levels - the last of which were sat in 2001 before the Curriculum 2000 reforms took effect.
Explaining what he meant by this, he said that shifts from year to year were quite normal but this year there had been a high number of changes in which grade boundaries had been moved significantly - he had highlighted those where six marks or more were involved.
Much of his report goes into explaining the somewhat technical nature of the process which has led him to his conclusions.
The new system
The new A-levels are in two parts, each made up of three units: AS-levels, followed by A2s in which students narrow their choice of subjects. They can also retake units to improve their grades.
The report says: "These design features might reasonably have been expected to lead to an increase, compared to the former "legacy" A-levels, in the proportion of full A-level candidates who achieved the A-level standard".
But: "Some aspects of the system were, from early in the implementation phase, causing concern to some people involved with the system."
And the evidence was that there was "no clear, consistent view" among exam board officials - and many examiners and teachers - about the standard required in the new AS and A2 units to ensure that the overall A-level standard was maintained.
This concern had been identified in relation to the AS-level in the review of Curriculum 2000 carried out last year by Professor David Hargreaves, then chief executive of the Qualifications and Curriculum Authority (QCA) - the regulator for the system in England.
That review was ordered by the Education Secretary, Estelle Morris, following the furore that surrounded the introduction of AS-levels, when there were numerous timetable clashes and complaints of exam overload.
"This clearly has created a risk that differing interpretations of the two standards exist," the report says.
"Such differing interpretations were strongly evident during my hearings and in written evidence."
Crucially, A2s had to be more demanding than AS-levels to reflect the different length of time students had been following their courses when they took the exams.
But Tomlinson says "there would appear to be no common understanding on how much greater the demands of A2 units should be".
QCA guidance on this was lacking - which "contributes significantly to a lack of common understanding among those involved in teaching and examining" the courses.
AS units were piloted to an extent - A2s were not, for reasons which were not clear.
So examiners had no scripts to use for comparison when it came to grading the exams this summer.
"This resulted, in part, from the speed of implementation of the policy as determined by ministers."
So not all the practical and statistical issues had been worked out in advance - resulting in this year's grading being what Mr Tomlinson described in his news conference as "an accident waiting to happen".
The grading process
Under the QCA's code of practice, for each unit in a given syllabus there is an "awarding meeting" in each exam board that looks at the marks candidates have been given and decides provisionally where to set two key boundaries.
These are the marks that will make up the boundary between grades A and B, and the boundary between grades E and U (a fail).
The other grades in between - C and D - are then filled in according to an arithmetical formula.
These meetings compare candidates' work to that from previous examinations, where possible, and review statistics provided by the exam board - which this year involved more data than usual, including for the first time predicted A-level grades based on candidates' previous GCSE results.
Each exam board has an "accountable officer" - typically the chief executive, who reviews the committee's boundary recommendations.
What happened this year
Tomlinson says he had evidence from three senior OCR examiners "that they believed that in making their recommendations they were expected to have very strong regard to the grade distributions" from the previous year's A-levels.
There was "a perceived pressure to deliver outcomes" in line with the old A-levels.
The QCA code of practice doesn't specify how much weight examiners should give to the different evidence they had before them - so they were not operating outside the code.
But he says the process within OCR "may have given undue weight" to past results and statistics - rather than candidates' actual performance.
In the case of the other boards - Edexcel and AQA - this apparently did not happen.
In OCR's case, however, a grades review process involved raising the boundaries in a "significant" number of course units - and the top official then raised them again.
"Evidence from chief examiners in geography, French, German, Spanish, history, psychology and government and politics expressed their concerns with these changes."
A total of 423 changes were made - in almost all cases to raise the grade boundaries. Some 86 of those were of six marks or more, the biggest change being 13 marks in one case.
Often the chief examiners for the subjects were not told.
The main impact was most likely to cut the pass rate, with a knock-on effect on all grades.
Tomlinson says OCR's accountable officer was "quite clear that his actions were entirely based on a view that the requirement to meet the A2 standard had been underestimated by some examiners".
Were the boards under pressure?
Tomlinson says the perennial public debate about A-levels - what he calls "background noise" - creates an ongoing pressure.
He was looking for evidence of more specific intervention.
There was a routine meeting in March between exam boards and the regulators in England, Wales and Northern Ireland - the Wales and Northern Ireland boards were not part of Tomlinson's remit.
The chief executives of the three English boards told Tomlinson their perception was that the QCA boss, Sir William Stubbs, was asking them "to give more emphasis than perhaps was proper to statistical data and the need to have an overall outcome similar to the 2001 legacy A-levels".
They were concerned enough to write to him asking for clarification - as Sir William himself revealed when he published the exchange of letters on Wednesday.
He wrote back saying "I do expect last year's A-level results to provide a very strong guide to this year's outcomes".
He added: "I am clear that grades for this summer's A-level candidates can only be determined using a combination of professional judgements and statistical evidence."
Tomlinson says that in OCR the perception remained - as an internal note revealed.
In more than one board, chairs of examiners felt pressure was being brought to bear to get this year's results close to last year's.
There was another big meeting on 26 July when the exam boards said the A-level results were revealing a rise in the pass rate and in the proportion of candidates getting the lowest grade.
Stubbs told Tomlinson he sought a reassurance that standards were being maintained, and indicated that an inquiry might be necessary - "a wholly proper" thing to have done as the regulator.
The chief executives had different impressions - but all believed they were being put under further pressure. For instance, Edexcel officials believed they were being threatened with an independent inquiry.
An OCR internal e-mail referred to its chief executive, Ron McClone, advising staff to award any remaining grades on the basis of candidates' predicted results from their GCSEs - the phrase used being that they should push the bottom grade boundary "until it squeaks".
Crucially, on the issue of political interference, Tomlinson says he has heard no evidence that ministers or education department officials "offered any pressure or guidance on the grading process or the final outcomes of this year's examinations".
"I therefore conclude that there was none."
He concludes that the QCA behaved properly and that the exam board chief executives "acted with integrity".
He says: "At the root of this is a longstanding misunderstanding of the difference between maintaining a standard and the proportion of candidates meeting that standard".
"This misunderstanding appears to exist at almost all levels of the system, and in society at large."
The BBC is not responsible for the content of external internet sites
Top Education stories now:
Links to more Education stories are at the foot of the page.
|E-mail this story to a friend|
Links to more Education stories
To BBC Sport>> | To BBC Weather>> | To BBC World Service>>
© MMIII | News Sources | Privacy