Last year we committed to do further work to evaluate the new GCSEs in French, German and Spanish, and to consider whether there was a case for making an adjustment to the grading. There are several strands to this work including – gathering teachers’ views on the new GCSEs, evaluating the way the new assessments have functioned, and considering how performance in the new GCSEs compares to the Common European Framework of Reference for languages (CEFR). We’re still analysing and completing our research on these strands but here are some headlines.
What do language teachers think of the new GCSEs?
In the autumn we spoke to a sample of teachers who had been teaching the new GCSEs in French, German and Spanish. This was really valuable in understanding what schools and colleges think of these new GCSEs.
- Teachers generally thought that the new linear structure, with all exams at the end of the course, was better for students’ language development.
- There was overwhelming support for the removal of controlled assessment.
- While teachers generally preferred the new GCSEs, there were concerns about some questions being too difficult, particularly in the listening and reading assessments.
As well as talking to teachers we have done some analysis to compare the 2018 assessments with those from the old GCSEs in 2017. The content of the new assessments was designed to be more demanding in line with the Government’s intention for the reformed GCSEs. However, ahead of the first assessments in 2018, there were concerns that changes such as the requirement for more questions to be in the target language, and listening material to be at ‘near-native’ speed, would make them overly difficult.
How did students perform in the new assessments?
We began by looking at the performance of students question by question, and we calculated a facility score for each question to indicate its level of difficulty. Looking at these scores can indicate if the questions are appropriately targeted to the students taking the assessment.
Our early analysis shows that overall, the 2018 assessments were more difficult than in 2017. However, in most cases this resulted from papers moving from being a bit too easy - based on the proportion of students obtaining full marks on each question - to within an 'ideal' range.
This increase in difficulty has produced a better spread of marks compared to the previous assessments, which often had candidates bunched at the top end of the mark distribution. This better spread of marks in 2018 meant that the grade boundaries were more spread out which, while retaining broadly the same proportion of students obtaining each grade as in 2017, gives greater confidence that students receive the grade their performance deserves.
What’s causing the change?
We then investigated what might have caused this increase in difficulty. To do this we examined features of the questions, with the help of subject experts, including question type, word count, language used in the question, demand of the vocabulary, and how likely students were to be familiar with the topic, among others.
The results suggest that the increased difficulty in 2018 is not caused by the features which teachers were worried about before the assessments were sat (inclusion of questions in the target language, speech speed or pause length). Instead, the increases in difficulty appear to be due to more demanding vocabulary in the reading and listening texts, and questions requiring more ‘work’ from students to answer the question; that is, not being able to rely on spotting key words or phrases. The introduction of extract-based questions, translation questions and more short answer questions (rather than multiple choice or matching type questions) is also likely to have increased demand.
What’s the impact of the removal of controlled assessment?
We also looked at how well the new GCSEs were functioning at the component level, particularly with the change of the writing assessment from a controlled assessment to an exam. In 2017, students obtained much higher marks on the writing controlled assessment than on the listening or reading papers. Our early analysis suggests that the balance between the components improved in 2018, with students generally performing similarly on each component. This will result in performance across the skills being better reflected in the overall grades received by students. Suggesting that, overall, the new GCSEs are functioning better.What next?
We will complete our research and analysis of the strands of work and report in detail, including any impact on grading, later in the year. In the meantime, students and teachers preparing for GCSEs in French, German and Spanish this summer can be reassured that our early analysis indicates the reformed qualifications are functioning well.
If you would like to talk to Ofqual about any of the issues raised in this blog, please contact us at firstname.lastname@example.org.
By Tim Stratton, Research Associate, Ofqual