https://ofqual.blog.gov.uk/2019/02/14/evaluating-new-gcses-in-french-german-and-spanish/

Evaluating new GCSEs in French, German and Spanish

Last year we committed to do further work to evaluate the new GCSEs in French, German and Spanish, and to consider whether there was a case for making an adjustment to the grading. There are several strands to this work including – gathering teachers’ views on the new GCSEs, evaluating the way the new assessments have functioned, and considering how performance in the new GCSEs compares to the Common European Framework of Reference for languages (CEFR). We’re still analysing and completing our research on these strands but here are some headlines.

What do language teachers think of the new GCSEs?

In the autumn we spoke to a sample of teachers who had been teaching the new GCSEs in French, German and Spanish. This was really valuable in understanding what schools and colleges think of these new GCSEs.

  • Teachers generally thought that the new linear structure, with all exams at the end of the course, was better for students’ language development.
  • There was overwhelming support for the removal of controlled assessment.
  • While teachers generally preferred the new GCSEs, there were concerns about some questions being too difficult, particularly in the listening and reading assessments.

As well as talking to teachers we have done some analysis to compare the 2018 assessments with those from the old GCSEs in 2017. The content of the new assessments was designed to be more demanding in line with the Government’s intention for the reformed GCSEs. However, ahead of the first assessments in 2018, there were concerns that changes such as the requirement for more questions to be in the target language, and listening material to be at ‘near-native’ speed, would make them overly difficult.

How did students perform in the new assessments?

We began by looking at the performance of students question by question, and we calculated a facility score for each question to indicate its level of difficulty. Looking at these scores can indicate if the questions are appropriately targeted to the students taking the assessment.

Our early analysis shows that overall, the 2018 assessments were more difficult than in 2017. However, in most cases this resulted from papers moving from being a bit too easy - based on the proportion of students obtaining full marks on each question - to within an 'ideal' range.

This increase in difficulty has produced a better spread of marks compared to the previous assessments, which often had candidates bunched at the top end of the mark distribution. This better spread of marks in 2018 meant that the grade boundaries were more spread out which, while retaining broadly the same proportion of students obtaining each grade as in 2017, gives greater confidence that students receive the grade their performance deserves.

What’s causing the change?

We then investigated what might have caused this increase in difficulty. To do this we examined features of the questions, with the help of subject experts, including question type, word count, language used in the question, demand of the vocabulary, and how likely students were to be familiar with the topic, among others.

The results suggest that the increased difficulty in 2018 is not caused by the features which teachers were worried about before the assessments were sat (inclusion of questions in the target language, speech speed or pause length). Instead, the increases in difficulty appear to be due to more demanding vocabulary in the reading and listening texts, and questions requiring more ‘work’ from students to answer the question; that is, not being able to rely on spotting key words or phrases. The introduction of extract-based questions, translation questions and more short answer questions (rather than multiple choice or matching type questions) is also likely to have increased demand.

What’s the impact of the removal of controlled assessment?

We also looked at how well the new GCSEs were functioning at the component level, particularly with the change of the writing assessment from a controlled assessment to an exam. In 2017, students obtained much higher marks on the writing controlled assessment than on the listening or reading papers. Our early analysis suggests that the balance between the components improved in 2018, with students generally performing similarly on each component. This will result in performance across the skills being better reflected in the overall grades received by students. Suggesting that, overall, the new GCSEs are functioning better.What next?

We will complete our research and analysis of the strands of work and report in detail, including any impact on grading, later in the year. In the meantime, students and teachers preparing for GCSEs in French, German and Spanish this summer can be reassured that our early analysis indicates the reformed qualifications are functioning well.

If you would like to talk to Ofqual about any of the issues raised in this blog, please contact us at public.enquiries@ofqual.gov.uk.

By Tim Stratton, Research Associate, Ofqual

9 comments

  1. Comment by Victoria posted on

    Written, obviously, by someone who has absolutely no clue about language teaching and learning. It's all very well to state that you listened to mfl teachers, but that is made redundant when you didn't bothering hearing us. The new GCSE's are a disgrace and turning students off from learning languages...Can we all really be wrong???

    Reply
    • Replies to Victoria>

      Comment by iandexter posted on

      Thank you for letting us know your thoughts on this blog post. Just to clarify, Ofqual is responsible for ensuring that the assessments validly and appropriately assess the required subject content (while the Department for Education develops specific content for each subject).

      In terms of the changes that fall within Ofqual’s remit we have, encouragingly, received feedback from teachers that indicates that the removal of controlled assessments for speaking and writing assessments, and the re-balancing of the four assessment objectives covering listening, speaking, reading and writing are positive changes that were welcomed by teachers and students alike. These changes were felt to contribute to more effective language learning overall.

      Nonetheless, some of the teachers that we spoke to raised individual points over aspects of the 2018 exams: we’ll be taking these up with the relevant exam boards.

      Reply
  2. Comment by Elizabeth Brazier posted on

    How can this research be accurate when you have no idea of the number of native speakers sitting these exams each year?

    Reply
    • Replies to Elizabeth Brazier>

      Comment by katekeating posted on

      Thank you for your comment. During 2017 we conducted <a href="https://www.gov.uk/government/publications/native-speakers-in-a-level-modern-foreign-languages">some research</a> into the impact of native speakers on grading in modern foreign language A levels. We found that the proportion of native speakers taking these A levels is relatively small. Given the larger numbers of learners taking GCSEs in modern foreign languages, the influence of native speakers on results in these exams could be even more limited. However, if exam boards came to us with evidence that these numbers had significantly changed, and the influence of native speakers was contributing to grading concerns, we would take a closer look. Additionally, the term ‘native speakers’ may cover a broad range of experience - while one student might be bilingual, another may benefit from having, for example, a French grandparent. It would not be appropriate to consider those two students in the same way.

      Reply
  3. Comment by Elizabeth Brazier posted on

    Thank you for your reply. I am puzzled about how you are able to identify native speakers or even those who are brought up as bilingual from other students as it is not recorded on the exam entry details.

    Reply
  4. Comment by Andrea O’Neill posted on

    The influence of native speakers at GCSE level will surely be higher than at A level given the smaller numbers taking A level languages anyway. As languages teachers contributing to this blog, we are fully aware of the high number of natives taking the exams and the grades they achieve compared to non natives. It would be useful for you to see what the uptake of MFL at universities is of natives compared to non natives. In addition to the impact that natives have on grading, the punitive marking at both GCSE and A level must be addressed as a matter of urgency if MFL are to be viable subjects in the future. National statistics have shown that students who take mfl at GCSE/ A level achieve a grade lower than in other subjects. Statistics show that THE SAME students who take a language and humanities at GCSE achieve at least one grade lower in their language on average than they do in the humanities subjects, across all ability ranges. This topic will not go away until all subjects are brought in line. It is becoming more difficult to advise students to take a language at GCSE/A level with the knowledge that they will likely achieve a lower grade than other subjects.

    Reply
    • Replies to Andrea O’Neill>

      Comment by katekeating posted on

      Thanks for your message Andrea. Subject experts have raised similar issues with us in the past. Because of this, we completed a comprehensive programme of research into inter-subject comparability in MFL A levels spanning the past 3 years. We spoke to the broadest possible range of interested parties, including higher education, subject experts, teachers and students. We published the outcome of this work, and our decision not to adjust grading standards in these A level subjects, in November. However, we recognise that perceived grading severity undermines confidence. Therefore, we have committed to working with the exam boards to ensure that these subjects do not become statistically more severely graded in the future. And we have confirmed we will now look at the evidence for a potential adjustment to grading standards in GCSE French, German and Spanish. As part of this work, we will consider the level of student performance in MFL subjects at key grades, to address the question of whether the performance at an adjusted (more lenient) grade 4 would still be acceptable.

      Reply
  5. Comment by Paul Smith posted on

    Whilst it may be the case that there are positive elements to the reformed GCSE, the grave concern that we have as language teachers is that students are looking at the new courses and voting definitively with their feet and opting for other subjects. There are parts of England and Wales where fewer than 30% of students are opting for languages. This is not an argument for "dumbing down", far from it, as I do feel that the new GCSE prepares very well for the new A level, however, we do have a duty as language teachers to ensure that ALL students feel that they are capable of gaining a respectable grade in languages. The listening comprehension papers in particular do not, I feel, allow students to demonstrate what they are able to understand and some of the scenarios in the speaking are, frankly, ridiculous. We cannot afford to leave up to 70% of our students behind as we are currently doing.

    Reply

Leave a comment

We only ask for your email address so we know you're a real person

By submitting a comment you understand it may be published on this public website. Please read our privacy notice to see how the GOV.UK blogging platform handles your information.