It’s the holiday season, but some of us at Ofqual spend it focusing on how the exam boards will set their standards in GCSEs, AS and A levels. We write the rules for awarding, monitor exam boards’ awarding as it happens, review the emerging results and keep a close eye on how they are received in schools. This year, we’ll also be looking carefully at how the new arrangements for reviews and appeals are being implemented. And just like planning a holiday, once one series is over, we start planning the next one.
We’ve been thinking about how to approach awarding the 13 new AS qualifications being taken this year since early 2014, well before teaching started. But what has that entailed? Mainly, the impact of decoupling the AS from the A level. Would fewer students take AS? Are they more or less able students? And how will the changes play out in individual school and college results?
On the first of these, we now know that fewer students are taking the new AS qualifications – entries have dropped by about 20% compared to 2015. The figure varies between subjects, but according to provisional entry data, English language and literature is down by just over 30%, while computing is down only 10%. See the slides from our summer 2016 symposium for more information.
We can model how changes in the ability of entry cohorts might impact results. For example, if there are far fewer students taking AS, but this year’s cohort had higher GCSE results, perhaps because weaker students were not being entered for AS, the predicted percentage of students achieving grade A would be higher to reflect that. On the other hand, if more able students are not entered for AS, the predictions for the percentage achieving grade A would be lower.
Our discussions with the exam boards about setting standards in the new AS awards have started with these statistical predictions – what’s become known as the ‘comparable outcomes’ approach. There are several reasons for this.
First, we know that students tend to do less well in the first year of a new qualification. That, in part, reflects the time it takes for teachers to become comfortable with new content and assessment design. Our comparable outcomes approach accommodates this. It means that students will not be disadvantaged by being the first cohort to sit these new linear AS qualifications.
Second, using predictions is the only way to make sure that grade standards between exam boards are in line, so that it is no easier to get a grade with one exam board than with another. We think that’s really important when students are competing for higher education or employment opportunities.
But exam boards won’t be slaves to the predictions; there may be other factors at play that have influenced outcomes. For example, we know that the GCSE results won’t tell us whether this year’s AS students are more or less motivated than in previous years. Therefore, senior examiners will look closely at the grade boundaries suggested by the predictions and they will be asked if those grade boundaries are acceptable. If necessary, the exam boards will look at a wider range of marks, student work and how particular papers performed. Exam boards will need to be able to justify any grade boundaries that are different from those suggested by the predictions.
So how might the changes affect individual school and college results? In recent years, we have published analyses of GCSE and A level results at individual school or college level but not at AS.
So ahead of the 2016 results, we’ve gone back to last summer and looked at the year-on-year variation in the percentage achieving grade A for schools and colleges entering students for AS.
The graphs show that, even when there are no changes to qualifications, individual schools and colleges see some year-on-year variation in their results. This can be due to many different factors, including differences in the mix of the students entered for particular subjects, different teaching approaches, changes in teaching staff or teaching time, and changes to qualifications. And as I’ve alluded to above, we know there is the potential for more variability as teachers and students get to grips with the new qualifications. So we’ll make sure exam boards set standards appropriately. We plan to publish our 2016 analysis on AS results day.
We will be watching how the results are received in schools, and in particular, the marking review requests that are submitted to exam boards. Many of you will know that the first changes to exam board marking review and appeal systems are being introduced this summer. In particular, marks will only be changed if there is a marking error. I want to emphasise that this is not a ‘clampdown’ on the number of reviews that can be requested by schools. It’s important that schools can have marks checked if they have concerns. So it won’t be harder to ask for a mark to be reviewed than in previous years. But schools will need to consider carefully the potential for a mark to be changed given the exam board fees associated with unsuccessful reviews.
Lastly, we are publishing a new report into awarding of last year’s (2015) modern foreign language (MFL) A levels in advance of this year’s results. In September 2014, we made some recommendations to exam boards so that their MFL A level qualifications could better differentiate between student abilities. The report identifies the actions each board took as a result of our recommendations. We reiterated our expectations regarding appropriate differentiation to exam boards ahead of the summer 2016 series and we will evaluate the impact of the changes in light of this year’s results.
Associate Director – Standards and Comparability
1 comment
Comment by Rodger Caseby posted on
Thanks for the update on the work you are doing. The exam boards have done a lot to help teachers understand the new qualifications (In my case AQA support for psychology), but the first year of a new specification is always a little uncertain. It's good you are taking steps to ensure this cohort are not unfairly penalised.