Introduction
A successful qualification reflects the achievements of its holder. It signifies the attainment of a specific set of skills. Those who rely on the qualification (for example, colleges or universities making admissions decisions, or employers deciding who to employ) need to be able to trust what it indicates. They may well be comparing people who have been awarded the same qualification in different years, so consistency between years is important. Equally, they may need to compare the same qualification offered by different exam boards.
Ofqual’s priority during summer awarding is to align standards between exam boards in a subject and over time. So it should be no easier, or difficult, to get a particular grade with one board than another and, all things being equal, a student who got a particular grade in a subject last year should get the same grade this year.
So how does what a student writes in their exam become a grade? There are two main stages, marking and awarding. Exam boards are responsible for marking and awarding and Ofqual oversees the process as the qualifications regulator.
Marking
In most subjects students take their exams in May and June each year. But preparations will have been going on for several years before those exams are sat. Senior examiners will have started drafting question papers up to two years before the exam date, and the exam boards will have been recruiting examiners to mark students’ work.
At the end of an exam, completed papers are sent to examiners for marking. Examiners are trained to mark to a required standard using a mark scheme, which provides guidance about how marks should be awarded. Before they can start marking, they have to show that they can mark a sample of ‘live’ answers to the required standard.
During the marking window, which is usually 3-4 weeks for each paper, each examiner's work is quality checked by their respective exam board to ensure their marking is consistent, and to the required standard. The types of check vary depending on whether scripts are marked on paper or electronically (on-screen), as well as whether they are marked by question or as a whole paper.
Where marking is done on-screen checking is done by including ‘seeds’ randomly through the marking. These ‘seeds’ are responses that senior examiners have previously reviewed and agreed a mark. Examiners do not know which items are ‘seeds’ and if they don’t mark them to the agreed standard, they can be stopped from marking until they have spoken to a more senior examiner.
Where scripts are marked on paper, examiners send samples of their marking to a more senior examiner for checking. If an examiner is not marking to the required standard, they are not allowed to continue and their scripts are given to a different examiner.
These many checks are performed to deliver marking that is consistent and fair across all candidates. However because students have the option to take papers produced by different boards, and additionally because the difficulty of papers will naturally vary by small amounts from one year to the next, a second process is required. It’s known as awarding.
Awarding
Once all exam scripts have been marked, grade boundaries are set. Exam boards convene awarding committees for each subject to recommend minimum marks for each unit of the qualification. Committees are chaired by a senior examiner who has overall responsibility for standards in that subject. The Committees also include a chief examiner, principal examiners and principal moderators, as well as exam board technical experts.
All exam boards must have awarding processes that meet Ofqual’s rules, which are set out in a code of practice. The basic principle is that if the group of students (the cohort) taking a qualification in one year is of similar ability to the cohort in the previous year then the overall results (outcomes) should be comparable. To do this, exam boards produce a reference matrix, based on the results of a previous cohort. This reference matrix, which compares Key Stage 2 attainment with the GCSE grades that this previous cohort achieved, is applied by each Exam Board to predict outcomes for the current cohort. For A levels, the starting position for predictions is the results achieved in GCSEs.
Committees use this evidence of how well the cohort performed in KS2 tests to predict the proportion of students that are likely to achieve, say, a grade C in GCSE mathematics, given previous experience of the statistical relationship. For AS and A level, performance at GCSE is used to predict likely proportions achieving key grades. Other factors are then considered, which include:
- Students’ work (marked exam papers and controlled assessment) – comparisons of scripts on grade boundaries from current and previous years.
- Reports from senior examiners/moderators about how the exam questions worked.
- Descriptions of the performance expected at key grades (A, C and F for GCSEs).
The use of data, to secure comparable outcomes, helps maintain standards particularly during a period of transition to a new or changed qualification (for example from modular to linear GCSEs). It also means that Ofqual can judge whether the standards are in line across exam boards. The predictions are intended to guide awarding at the cohort (national) level and not predict the grades that individual candidates will receive. Individual students may do better, or worse, than their prior attainment would suggest for a number of reasons.
Before any results are issued, exam boards send their expected results to Ofqual. Exam boards must provide evidence to justify any circumstances where the expected results are markedly different from the predictions. Ofqual will either accept the explanation provided by exam boards or challenge the results if the argument is not backed by sufficient evidence. As a result, significant year-on-year changes in performance at the national level are not likely. However if the exam boards have evidence of genuine improvement, the comparable outcomes approach allows that to be reflected in results.
Phil Beach
Director of Strategic Relationships, General Qualifications