REDUCING BURDENS ON THE SYSTEM
Standards over time
23. Ever since the 1996 Standards Over Time study undertaken by QCA's predecessor body, SCAA, and OFSTED, QCA has sought input from independent experts to its work and to verify its conclusions on the maintenance of standards over time. Such work has in the past been hampered by the absence of definitive evidence about students' performance over the past three decades. For more recent years, the systematic archiving of examination material will ensure that such evidence is available to inform future work.
24. Earlier this year, an independent report by a panel of three international experts, Chaired by Professor Eva Baker, commended QCA's work in monitoring A level standards, whilst also highlighting the difficulty of answering with certainty the question of whether standards have been maintained.
25. Whilst changes to qualification structure, syllabus content and assessment methods can make the task more difficult, I believe it is important that QCA should maintain efforts to investigate comparability from year to year and across the A level system, and should do so in a way that commands credibility with education professionals and the public. In doing so, QCA should build both on its existing rolling programme of five-yearly subject reviews and its use of external expertise and verification. The annual debate about A level standards undermines and devalues demonstrable improvements in teaching and learning over time, and the hard work and commitment of students. Action is needed to ensure that the evidence is available to rebut ill-founded claims both about the value of qualifications and about year-on-year comparisons. But, equally, it is vital that where legitimate concerns exist about standards, these must be looked into and appropriate remedial action identified and taken.
Setting and Maintaining the standard
26. The crucial first step is to establish a common understanding of what the standards should be. In my initial report, I highlighted the absence of a clear understanding of the standards or levels of demand for either AS or A2 assessment which lay at the heart of the concerns this year about the grading of A level performance.
27. QCA, the other regulators and the awarding bodies have been working to rectify this deficiency. By the January 2003 examinations there will be in place:
- a description in broad terms of the separate AS and A2 standards and levels of demand, and their relationship with the overall standard of the former A level system. This makes it clear that the level of AS is equivalent to the first half of a two year traditional A level course, and that A2 covers the more demanding material found in the second half;
- descriptions of the performance that merits an E grade and A grade for both AS and A2 in the 27 A level subjects for which QCA has established subject criteria;
- exemplar material illustrating, through students' actual scripts and other material, the level of performance at A and E grades for both AS and A2 in the twelve most popular A level subjects, covering some 90% of the January entries.
28. By June 2003 the exemplar material will be extended to cover all A level subjects. In the longer term the material will be strengthened, refined and reviewed in the light of further experience.
29. I now recommend that QCA should establish an independent committee whose role would be to review and, if necessary, advise QCA publicly on whether or not standards are being maintained - advising on a limited number of subjects each year - using all the available evidence including subject syllabuses, students' work, mark schemes and question papers. The group should also be able to review and verify other aspects of QCA's regulatory work, as requested by the QCA board. This committee will help provide reassurance that standards are being kept continuously under review and that, where necessary, action will be identified and taken to safeguard standards over time.
30. I am satisfied that these measures will provide a secure foundation for the grading of A level entries; help ensure that individual candidates' performance can be consistently and reliably graded; and provide credible ongoing evidence that standards are being maintained.
31. In the longer term, however, I believe that there are some further steps which can reduce the complexity of the AS/A2 system and provide a more readily understood basis for grading.
Relationship between AS and A2
32. The annex to this report shows the key steps in the examining and grading process. Students' scripts and other work for AS and A2 units are marked and graded separately and then combined to determine the candidate's overall A level grade. The process to arrive at the final grade is complex.
33. First, a sample of students' work and other information is used to determine the mark which should define each grade boundary for each unit. Those grade boundaries are then translated into fixed points on a uniform mark scale (UMS). The range covered by this scale varies between units. For a 0-100 scale, the ranges of scores for each grade are: E=40-49, D=50-59, C=60-69, B=70-79, A=80-100. The scores for each grade boundary on scales which are less or greater than 100 will be proportionately lower or higher. From that, individual students' marks for each unit are translated into a UMS score for the unit. Marks which lie between two grade boundaries on the raw mark scale will translate into intermediate points between the same two boundaries on the UMS scale. The UMS scores for individual units are then added together to give an overall UMS score for the subject as a whole. The maximum total UMS score is always 600. The final A level grade is awarded based on where the overall score lies in relation to fixed points on this 600 point scale (ie E=240 to 299, D=300 to 359…).
34. This summer's experience, reinforced by much of the evidence submitted to my inquiry, demonstrates clearly the lack of clarity among teachers, students and parents about the way in which students' marks for individual units are converted into UMS scores for each unit and overall grades. In particular, there is no clear understanding of the relationship between marks awarded for individual units by markers and the UMS scores on which the final grade is based.
35. A further layer of complexity is added by the range of choice with which students are faced about the options for re-sitting units and "cashing-in" their AS units in exchange for an AS award part-way through their A level course. Broadly, after a student has completed the three AS units they may:
- exchange their unit UMS scores for an AS qualification in the relevant subject - an option known as "cashing-in";
- elect to move straight on to A2 units without cashing in their AS units; or
- re-sit some or all of the AS units in an attempt to improve either their AS grade before cashing-in or their overall A level grade.
36. Even after AS units have been cashed-in, the student may re-sit some or all of the same AS units in order to improve their overall A level grade.
37. These arrangements give students a great deal of flexibility. But the awarding bodies' current guidance on the 'entry, aggregation and certification' of A levels acknowledges that: "For many centres and students the decision whether or not to cash in can be a difficult one." That appraisal is certainly borne out by the remainder of the guidance, which, for instance, offers as a worked example a situation in which for a single subject the student sits 4 units (ie including 1 re-sit) to achieve their 3 unit AS award and a total of 11 (including 5 re-sits) to complete their full 6 unit A level course. One of the relevant AS units is repeated 3 times.
38. There are other potential complications. For example:
- if an AS award is cashed in at the end of the first year, the AS result must be declared on any UCAS application form. If they are not cashed in, the student may report the unit results to UCAS but does not have to;
- because a student may re-sit AS units after cashing-in, the AS unit results which comprise the original AS award may be different from the AS unit results which contribute to the overall A level grade.
39. None of this implies that the system is technically flawed. I am satisfied that operated in this way, and applying the correct AS and A2 standards, assessment will be robust, and that students will receive grades that are technically accurate and which reflect their overall performance. Action is needed over time to simplify the awarding arrangements and reduce the complexity and lack of transparency which affects perceptions about the system's reliability and fairness.
40. This might partly be rectified by more intensive efforts to provide accessible information about the grading process and the options open to students. But as it stands the system is unlikely ever to attract the levels of public and professional understanding which would prevent recurring confusion and dissatisfaction. In my view, the complexity of the current arrangements will continue to undermine the extent to which A level results are understood and trusted, even though the actual outcomes accurately reflect students' achievement.
41. Nor am I persuaded that the existing processes for sitting and re-sitting units and for combining AS and A2 scores are necessary to the effective accreditation of students' achievement. The rationale initially has been to provide a single overall grade which is directly comparable to those awarded for the former A level. As the new system becomes embedded and stands in its own right, the need to relate all current achievement to the previous A level outcomes will diminish. In effect the continuity of A level standards would be carried forward in the separate AS and A2 standards and levels of demand and the design of the qualifications rather than in direct comparability between students' past and present results.
42. I therefore recommend that in time, and as part of the 14-19 policy development process, the AS/A2 system should be simplified as follows:
- AS and A2 should be assessed, graded and awarded separately. They should become distinct and separate qualifications;
- as now, AS should cover the first half of a two year A level programme; and A2 should cover the more demanding second half. No element of assessment of either should be at a level which would have been outside the scope of a traditional A level course;
- before any new arrangements are introduced, the separate standards for AS and A2 should be securely established and embedded within the existing A level framework; and time must also be allowed for the necessary design, development and testing of the new arrangements, and of any further changes along the lines illustrated in paragraph 46, below.
As an interim measure, in the shorter term, I recommend that the QCA and Ministers should look urgently at the scope for simplifying the rules governing re-sits and the cashing-in of AS units, with a view to introducing changes for students embarking upon AS in September 2003 for examinations in 2004.
43. Separation of AS and A2 would build on the existing system. Examinations could continue to be set, taught, assessed and graded as now, using the established AS and A2 standards. It would also retain the consistency in the levels of demand inherited from the "legacy" A level which the current AS/A2 design is intended to secure, and it would retain the possibility of students receiving a qualification for the subjects which they drop after AS. At the same time, it would remove the complexities and scope for confusion within the process for re-sitting units, cashing-in, and combining AS and A2 unit results to give a single overall grade.
44. There are nonetheless some important policy and design issues which would need to be resolved, including:
- the extent to which completion of a relevant AS course should be a precondition of entry onto A2 courses;
- the extent to which A2 should include "synoptic" assessment - whether it should not only assess the material covered in the A2 units but also ensure that students can relate what they have learned across both the AS and A2;
- retaining flexibility for adults and other non-traditional learners;
- ensuring consistency with the structure and evolution of vocational A levels;
- maintaining the value and currency of the AS award in its own right.
45. Separating the two awards in this way would also provide an opportunity to review other aspects of the syllabus and assessment methods, for instance to reflect the nature of learning in specific subjects; to reduce burdens on the examination system by changing the weight of, and balance between, coursework and examinations; or to change the ways in which synoptic assessment is integrated into the course. The factors that would influence final decisions on these issues go well beyond the boundaries of my remit. I make no specific recommendations on them except that in following up the recommendation for separate AS and A2 awards, consideration should be given to these issues.
46. There is a range of potentially significant changes arising from this work. Reform should not be rushed. Implementation should take account of emerging timetables for wider 14-19 policy development. Equally crucially, sufficient time must be allowed for the necessary design, development and testing, and for schools and colleges to familiarise themselves with any changes. In my view it should be at least five years before significant changes are fully introduced, although the preparatory work could begin almost immediately.
Relevance for Northern Ireland and Wales
Maintenance of the national qualifications framework covering all three countries requires application of the A level standard and design changes on a three country basis. The recommendations in this chapter should therefore be taken forward jointly.