BBCi CATEGORIES   TV   RADIO   COMMUNICATE   WHERE I LIVE   INDEX    SEARCH 

Front Page | In depth | Education
INTERIM REPORT FINAL REPORT PART 1 PART 2 PART 3
FINAL REPORT - PART 1


Most computers will open this document automatically. You may need to download Adobe Acrobat Reader to open this PDF document.


On 3 December the full report of the Tomlinson inquiry into this year's A-level results was published. Read the full text of the report below.

SUMMARY OF CONCLUSIONS AND MAJOR RECOMMENDATIONS

1. This inquiry has sought to resolve the major concerns expressed this summer with the grading of A levels and propose arrangements which will secure the examinations in the future and provide assurances that the A level standard is being maintained year on year.

2. I remain convinced that my interim report and the subsequent review of grade boundaries dealt effectively with the major concerns and allegations about manipulation of the grading process. Some of the remaining concerns are not a consequence of grading but relate to subject syllabuses (technically known as "specifications") or the marking and moderation of students' work. These concerns are being dealt with separately by the QCA and the relevant awarding bodies.

3. Action by the QCA and the other regulatory bodies on my earlier recommendations, allied to further proposals made in this final report will, in my view, secure the standards and integrity of next year's examinations. That has been, and remains, my priority. The action taken includes defining the standard and levels of demand to be associated with the AS and A2 examinations, supported by exemplification of performance at grades A and E. This will, I believe, deal with the major concern I had with implementation of the Curriculum 2000 reforms.

4. Changes to the Code of Practice will ensure in future that a more appropriate balance is struck than was always the case in the past between professional judgement and statistical data in setting mark grade boundaries, and that late changes to these boundaries will require either the agreement of Chairs of Examiners or a report to QCA and the governing body of the relevant awarding body. Through these and other measures in hand, I hope that teachers, students, parents and users of A levels will have their confidence restored. However, it is vital that all the changes are communicated effectively and speedily to all the relevant groups; if nothing else, this inquiry has revealed the need for far better communication of changes to our qualification system. In this context I fully endorse the communications plans which the QCA is currently developing to improve professional and public understanding.

5. Once the above changes have been implemented over the coming months, a period of consolidation is necessary before further evolution of the AS and A2 system is undertaken. Any further changes must be carefully planned, piloted and introduced over a sensible period. My recommendations must be seen as evolutionary and should be considered in many cases within existing policy developments, including reduction of bureaucratic burdens on schools and colleges and the 14-19 proposals due to be published shortly.

6. The major recommendations cover:

    Longer term

  • De-coupling of AS and A2 to create two free-standing qualifications as part of the 14-19 policy developments. Consideration should be given at the same time to other changes in the design of A level assessment (paragraphs 42 and 53).

  • Further work on the practicality of introducing a post-qualifications admission (PQA) system for entry to Higher Education (paragraph 59).

7. This inquiry arose out of concerns expressed by schools in England about the processes followed by Awarding Bodies based in England. However, a number of my recommendations have impact and relevance not just in England but also in Wales and Northern Ireland. It is vitally important that the QCA and its partner bodies, ACCAC in Wales and CCEA in Northern Ireland, continue to work closely together to ensure the smooth and consistent delivery of qualifications throughout the three countries.

8. Finally I believe it to be vital that there is greater public understanding of the examination process and that as a consequence there is an end to the annual argument about A level results. The standard has not been lowered if an increased proportion of students meet it as a consequence of improved teaching and hard work.

^^ Back to Top

INTRODUCTION

9. This inquiry was set up following concerns expressed about the setting of A level standards in July 2002. The terms of reference for the inquiry are:

10. The first point was covered in my interim report published on 27 September. This led to the review of grades awarded to candidates for one or more units in 31 separate A level subjects. The outcomes of the review were that 9,800 candidate entries had unit grades improved and of these 1,945 candidates received higher overall AS and A level grades in at least one subject. Other recommendations in the interim report have been the subject of work by QCA, supported by a programme board. I have been encouraged by the progress made so far.

11. This final report deals primarily with point 2 of the remit though necessary reference has been made to the action proposed in relation to the recommendations made in my interim report. The priority of this final report has been to secure the examination process for 2003 so that there can be no repetition of the concerns expressed this year. In addition, proposals are made for the medium term once the standards and levels of demand of AS and A2 have been firmly established in the system and the revised arrangements in the Code of Practice have become fully operational. There should be no major upheavals of Curriculum 2000 before the above have been achieved.

12. The medium and longer term changes need to be considered as part of other significant policy developments such as the drive to reduce bureaucracy and simplify demands on schools and colleges and the 14 -19 proposals due to be announced shortly. The latter provides an ideal basis for considering a number of my recommendations.

13. I have been greatly helped in preparing this final report by the many written submissions from bodies and individuals (over 100) and by the three regional meetings I have held with teachers, head teachers, college principals, governors, pupils and students. A schedule of evidence presented to the inquiry is included as an annex. The views offered have always been constructive and insightful and I am grateful to each and everyone for their input, particularly given the pressures upon their time.

14. These meetings have also raised some continuing concerns with the 2002 examination process. A significant number of these relate to the quality of marking, coursework, promptness and extent of responses of the Awarding Bodies and to the effectiveness of the assessment requirements in some subjects. The latter are being revised for 2003 and some other concerns are covered in this final report.

15. Many of my recommendations will involve not only the Awarding Bodies in England but also those in Wales and Northern Ireland. I have appreciated the cooperation of colleagues from ACCAC in Wales and CCEA in Northern Ireland.

16. I also wish to record my sincere thanks to members of the reference group: Mike Cresswell, Sue Kirkham, Judith Norrington, David Raffe, Sue Singer and Leslie Wagner. Together they have provided a sounding board and have interrogated the evidence. This report is, however, mine. None of the views, nor any flaws, are attributable to the group. Finally I must thank the team who have supported my inquiry: Audrey Beckford, Marcus Chrysostomou, Darren Goff, Nancy McLean, Georgina Nolan, Peter O'Connor, Kate Taylor and Matthew White.

^^ Back to Top

THE PURPOSE AND DESIGN OF GCE A LEVELS

What are A levels for?

17. Ever since their introduction, A levels have been associated with entry to higher education. This remains a valid and useful application. But over time they have also acquired a broader significance as a precursor to employment and as one strand in a qualifications framework which is designed to recognise the full range of advanced achievement of which young people are capable, ranging from the purely academic and theoretical learning through to the skills and knowledge associated with specific jobs. This trend is embodied in the Curriculum 2000 reforms which increased the flexibility of, and broadened the range of subjects and types of learning within, the A level strand, for instance by establishing A levels in vocational subjects.

18. During my inquiry, I have found broad commitment to this wider purpose of A levels as a means of recognising young people's achievement, and strong support for the principles of the Curriculum 2000 reforms. Students in particular value the possibility of studying a broad range of subjects in their first year of A levels and achieving a recognised AS qualification at that point for those subjects which they do not pursue to full A level.

19. There is also strong support for the existing A level design principle that the achievement required for an A level should remain the same from year to year and reflect predetermined standards of attainment, irrespective of how many students achieve the necessary standards. This is often thought of as "criterion referencing", although paragraphs 63 to 65 describe some of the difficulties of applying pure criterion referencing to A level examinations and assessment. I have encountered very little systematic support for a return to grading in which fixed quotas of grades would be awarded to students according to rank order rather than performance against a fixed standard of achievement (broadly, "norm referencing"). This conclusion is not incompatible with their traditional role as a filter for entry into higher education or employment. But it nonetheless has significant implications for the design of the A level framework.

20. Even in relation to university entry, A levels must constitute more than a simple means of ranking students to help HE admissions authorities choose between applicants. They must also contain an assurance that students have acquired the specific skills and knowledge that they need in order to embark on their chosen degree course.

21. This role of accrediting specific levels of knowledge and skills is even more important if A levels are to act as a consistent and reliable indicator of students' achievements from one cohort to the next. In this wider context it is the accreditation of known standards of achievement which is important rather than the ranking of individual students against their peers. This reinforces the weight of argument in favour of judging students' performance against a fixed standard. If 100% of students reach the standard then 100% should pass, and that outcome should not be seen as a 'lowering of standards'.

22. The design implications arising from this have been significant in the recent developments of the A level system. Confident matching of students' performance to a fixed standard requires not just accurate and consistent marking but also the ability to define and maintain standards of grading from year to year and between subjects, and the means to judge the performance of individual candidates against that standard.

^^ Back to Top

REDUCING BURDENS ON THE SYSTEM

Standards over time

23. Ever since the 1996 Standards Over Time study undertaken by QCA's predecessor body, SCAA, and OFSTED, QCA has sought input from independent experts to its work and to verify its conclusions on the maintenance of standards over time. Such work has in the past been hampered by the absence of definitive evidence about students' performance over the past three decades. For more recent years, the systematic archiving of examination material will ensure that such evidence is available to inform future work.

24. Earlier this year, an independent report by a panel of three international experts, Chaired by Professor Eva Baker, commended QCA's work in monitoring A level standards, whilst also highlighting the difficulty of answering with certainty the question of whether standards have been maintained.

25. Whilst changes to qualification structure, syllabus content and assessment methods can make the task more difficult, I believe it is important that QCA should maintain efforts to investigate comparability from year to year and across the A level system, and should do so in a way that commands credibility with education professionals and the public. In doing so, QCA should build both on its existing rolling programme of five-yearly subject reviews and its use of external expertise and verification. The annual debate about A level standards undermines and devalues demonstrable improvements in teaching and learning over time, and the hard work and commitment of students. Action is needed to ensure that the evidence is available to rebut ill-founded claims both about the value of qualifications and about year-on-year comparisons. But, equally, it is vital that where legitimate concerns exist about standards, these must be looked into and appropriate remedial action identified and taken.

Setting and Maintaining the standard

26. The crucial first step is to establish a common understanding of what the standards should be. In my initial report, I highlighted the absence of a clear understanding of the standards or levels of demand for either AS or A2 assessment which lay at the heart of the concerns this year about the grading of A level performance.

27. QCA, the other regulators and the awarding bodies have been working to rectify this deficiency. By the January 2003 examinations there will be in place:

28. By June 2003 the exemplar material will be extended to cover all A level subjects. In the longer term the material will be strengthened, refined and reviewed in the light of further experience.

29. I now recommend that QCA should establish an independent committee whose role would be to review and, if necessary, advise QCA publicly on whether or not standards are being maintained - advising on a limited number of subjects each year - using all the available evidence including subject syllabuses, students' work, mark schemes and question papers. The group should also be able to review and verify other aspects of QCA's regulatory work, as requested by the QCA board. This committee will help provide reassurance that standards are being kept continuously under review and that, where necessary, action will be identified and taken to safeguard standards over time.

30. I am satisfied that these measures will provide a secure foundation for the grading of A level entries; help ensure that individual candidates' performance can be consistently and reliably graded; and provide credible ongoing evidence that standards are being maintained.

31. In the longer term, however, I believe that there are some further steps which can reduce the complexity of the AS/A2 system and provide a more readily understood basis for grading.

Relationship between AS and A2

32. The annex to this report shows the key steps in the examining and grading process. Students' scripts and other work for AS and A2 units are marked and graded separately and then combined to determine the candidate's overall A level grade. The process to arrive at the final grade is complex.

33. First, a sample of students' work and other information is used to determine the mark which should define each grade boundary for each unit. Those grade boundaries are then translated into fixed points on a uniform mark scale (UMS). The range covered by this scale varies between units. For a 0-100 scale, the ranges of scores for each grade are: E=40-49, D=50-59, C=60-69, B=70-79, A=80-100. The scores for each grade boundary on scales which are less or greater than 100 will be proportionately lower or higher. From that, individual students' marks for each unit are translated into a UMS score for the unit. Marks which lie between two grade boundaries on the raw mark scale will translate into intermediate points between the same two boundaries on the UMS scale. The UMS scores for individual units are then added together to give an overall UMS score for the subject as a whole. The maximum total UMS score is always 600. The final A level grade is awarded based on where the overall score lies in relation to fixed points on this 600 point scale (ie E=240 to 299, D=300 to 359).

34. This summer's experience, reinforced by much of the evidence submitted to my inquiry, demonstrates clearly the lack of clarity among teachers, students and parents about the way in which students' marks for individual units are converted into UMS scores for each unit and overall grades. In particular, there is no clear understanding of the relationship between marks awarded for individual units by markers and the UMS scores on which the final grade is based.

35. A further layer of complexity is added by the range of choice with which students are faced about the options for re-sitting units and "cashing-in" their AS units in exchange for an AS award part-way through their A level course. Broadly, after a student has completed the three AS units they may:

  • exchange their unit UMS scores for an AS qualification in the relevant subject - an option known as "cashing-in";

  • elect to move straight on to A2 units without cashing in their AS units; or

  • re-sit some or all of the AS units in an attempt to improve either their AS grade before cashing-in or their overall A level grade.

36. Even after AS units have been cashed-in, the student may re-sit some or all of the same AS units in order to improve their overall A level grade.

37. These arrangements give students a great deal of flexibility. But the awarding bodies' current guidance on the 'entry, aggregation and certification' of A levels acknowledges that: "For many centres and students the decision whether or not to cash in can be a difficult one." That appraisal is certainly borne out by the remainder of the guidance, which, for instance, offers as a worked example a situation in which for a single subject the student sits 4 units (ie including 1 re-sit) to achieve their 3 unit AS award and a total of 11 (including 5 re-sits) to complete their full 6 unit A level course. One of the relevant AS units is repeated 3 times.

38. There are other potential complications. For example:

  • if an AS award is cashed in at the end of the first year, the AS result must be declared on any UCAS application form. If they are not cashed in, the student may report the unit results to UCAS but does not have to;

  • because a student may re-sit AS units after cashing-in, the AS unit results which comprise the original AS award may be different from the AS unit results which contribute to the overall A level grade.

39. None of this implies that the system is technically flawed. I am satisfied that operated in this way, and applying the correct AS and A2 standards, assessment will be robust, and that students will receive grades that are technically accurate and which reflect their overall performance. Action is needed over time to simplify the awarding arrangements and reduce the complexity and lack of transparency which affects perceptions about the system's reliability and fairness.

40. This might partly be rectified by more intensive efforts to provide accessible information about the grading process and the options open to students. But as it stands the system is unlikely ever to attract the levels of public and professional understanding which would prevent recurring confusion and dissatisfaction. In my view, the complexity of the current arrangements will continue to undermine the extent to which A level results are understood and trusted, even though the actual outcomes accurately reflect students' achievement.

41. Nor am I persuaded that the existing processes for sitting and re-sitting units and for combining AS and A2 scores are necessary to the effective accreditation of students' achievement. The rationale initially has been to provide a single overall grade which is directly comparable to those awarded for the former A level. As the new system becomes embedded and stands in its own right, the need to relate all current achievement to the previous A level outcomes will diminish. In effect the continuity of A level standards would be carried forward in the separate AS and A2 standards and levels of demand and the design of the qualifications rather than in direct comparability between students' past and present results.

42. I therefore recommend that in time, and as part of the 14-19 policy development process, the AS/A2 system should be simplified as follows:

  • AS and A2 should be assessed, graded and awarded separately. They should become distinct and separate qualifications;

  • as now, AS should cover the first half of a two year A level programme; and A2 should cover the more demanding second half. No element of assessment of either should be at a level which would have been outside the scope of a traditional A level course;

  • before any new arrangements are introduced, the separate standards for AS and A2 should be securely established and embedded within the existing A level framework; and time must also be allowed for the necessary design, development and testing of the new arrangements, and of any further changes along the lines illustrated in paragraph 46, below.

As an interim measure, in the shorter term, I recommend that the QCA and Ministers should look urgently at the scope for simplifying the rules governing re-sits and the cashing-in of AS units, with a view to introducing changes for students embarking upon AS in September 2003 for examinations in 2004.

43. Separation of AS and A2 would build on the existing system. Examinations could continue to be set, taught, assessed and graded as now, using the established AS and A2 standards. It would also retain the consistency in the levels of demand inherited from the "legacy" A level which the current AS/A2 design is intended to secure, and it would retain the possibility of students receiving a qualification for the subjects which they drop after AS. At the same time, it would remove the complexities and scope for confusion within the process for re-sitting units, cashing-in, and combining AS and A2 unit results to give a single overall grade.

44. There are nonetheless some important policy and design issues which would need to be resolved, including:

  • the extent to which completion of a relevant AS course should be a precondition of entry onto A2 courses;

  • the extent to which A2 should include "synoptic" assessment - whether it should not only assess the material covered in the A2 units but also ensure that students can relate what they have learned across both the AS and A2;

  • retaining flexibility for adults and other non-traditional learners;

  • ensuring consistency with the structure and evolution of vocational A levels;

  • maintaining the value and currency of the AS award in its own right.

45. Separating the two awards in this way would also provide an opportunity to review other aspects of the syllabus and assessment methods, for instance to reflect the nature of learning in specific subjects; to reduce burdens on the examination system by changing the weight of, and balance between, coursework and examinations; or to change the ways in which synoptic assessment is integrated into the course. The factors that would influence final decisions on these issues go well beyond the boundaries of my remit. I make no specific recommendations on them except that in following up the recommendation for separate AS and A2 awards, consideration should be given to these issues.

46. There is a range of potentially significant changes arising from this work. Reform should not be rushed. Implementation should take account of emerging timetables for wider 14-19 policy development. Equally crucially, sufficient time must be allowed for the necessary design, development and testing, and for schools and colleges to familiarise themselves with any changes. In my view it should be at least five years before significant changes are fully introduced, although the preparatory work could begin almost immediately.

Relevance for Northern Ireland and Wales

Maintenance of the national qualifications framework covering all three countries requires application of the A level standard and design changes on a three country basis. The recommendations in this chapter should therefore be taken forward jointly.

^^ Back to Top
 © MMV | News Sources | Privacy