Last year we committed to do further work to evaluate the new GCSEs in French, German and Spanish, and to consider whether there was a case for making an adjustment to the grading. There are several strands to this work including – gathering teachers’ views on the new GCSEs, evaluating the way the new assessments have functioned, and considering how performance in the new GCSEs compares to the Common European Framework of Reference for languages (CEFR). We’re still analysing and completing our research on these strands but here are some headlines.
What do language teachers think of the new GCSEs?
In the autumn we spoke to a sample of teachers who had been teaching the new GCSEs in French, German and Spanish. This was really valuable in understanding what schools and colleges think of these new GCSEs.
- Teachers generally thought that the new linear structure, with all exams at the end of the course, was better for students’ language development.
- There was overwhelming support for the removal of controlled assessment.
- While teachers generally preferred the new GCSEs, there were concerns about some questions being too difficult, particularly in the listening and reading assessments.
As well as talking to teachers we have done some analysis to compare the 2018 assessments with those from the old GCSEs in 2017. The content of the new assessments was designed to be more demanding in line with the Government’s intention for the reformed GCSEs. However, ahead of the first assessments in 2018, there were concerns that changes such as the requirement for more questions to be in the target language, and listening material to be at ‘near-native’ speed, would make them overly difficult.
How did students perform in the new assessments?
We began by looking at the performance of students question by question, and we calculated a facility score for each question to indicate its level of difficulty. Looking at these scores can indicate if the questions are appropriately targeted to the students taking the assessment.
Our early analysis shows that overall, the 2018 assessments were more difficult than in 2017. However, in most cases this resulted from papers moving from being a bit too easy - based on the proportion of students obtaining full marks on each question - to within an 'ideal' range.
This increase in difficulty has produced a better spread of marks compared to the previous assessments, which often had candidates bunched at the top end of the mark distribution. This better spread of marks in 2018 meant that the grade boundaries were more spread out which, while retaining broadly the same proportion of students obtaining each grade as in 2017, gives greater confidence that students receive the grade their performance deserves.
What’s causing the change?
We then investigated what might have caused this increase in difficulty. To do this we examined features of the questions, with the help of subject experts, including question type, word count, language used in the question, demand of the vocabulary, and how likely students were to be familiar with the topic, among others.
The results suggest that the increased difficulty in 2018 is not caused by the features which teachers were worried about before the assessments were sat (inclusion of questions in the target language, speech speed or pause length). Instead, the increases in difficulty appear to be due to more demanding vocabulary in the reading and listening texts, and questions requiring more ‘work’ from students to answer the question; that is, not being able to rely on spotting key words or phrases. The introduction of extract-based questions, translation questions and more short answer questions (rather than multiple choice or matching type questions) is also likely to have increased demand.
What’s the impact of the removal of controlled assessment?
We also looked at how well the new GCSEs were functioning at the component level, particularly with the change of the writing assessment from a controlled assessment to an exam. In 2017, students obtained much higher marks on the writing controlled assessment than on the listening or reading papers. Our early analysis suggests that the balance between the components improved in 2018, with students generally performing similarly on each component. This will result in performance across the skills being better reflected in the overall grades received by students. Suggesting that, overall, the new GCSEs are functioning better.
What next?
We will complete our research and analysis of the strands of work and report in detail, including any impact on grading, later in the year. In the meantime, students and teachers preparing for GCSEs in French, German and Spanish this summer can be reassured that our early analysis indicates the reformed qualifications are functioning well.
If you would like to talk to Ofqual about any of the issues raised in this blog, please contact us at public.enquiries@ofqual.gov.uk.
By Tim Stratton, Research Associate, Ofqual
19 comments
Comment by Victoria posted on
Written, obviously, by someone who has absolutely no clue about language teaching and learning. It's all very well to state that you listened to mfl teachers, but that is made redundant when you didn't bothering hearing us. The new GCSE's are a disgrace and turning students off from learning languages...Can we all really be wrong???
Comment by iandexter posted on
Thank you for letting us know your thoughts on this blog post. Just to clarify, Ofqual is responsible for ensuring that the assessments validly and appropriately assess the required subject content (while the Department for Education develops specific content for each subject).
In terms of the changes that fall within Ofqual’s remit we have, encouragingly, received feedback from teachers that indicates that the removal of controlled assessments for speaking and writing assessments, and the re-balancing of the four assessment objectives covering listening, speaking, reading and writing are positive changes that were welcomed by teachers and students alike. These changes were felt to contribute to more effective language learning overall.
Nonetheless, some of the teachers that we spoke to raised individual points over aspects of the 2018 exams: we’ll be taking these up with the relevant exam boards.
Comment by Elizabeth Brazier posted on
How can this research be accurate when you have no idea of the number of native speakers sitting these exams each year?
Comment by katekeating posted on
Thank you for your comment. During 2017 we conducted some research into the impact of native speakers on grading in modern foreign language A levels. We found that the proportion of native speakers taking these A levels is relatively small. Given the larger numbers of learners taking GCSEs in modern foreign languages, the influence of native speakers on results in these exams could be even more limited. However, if exam boards came to us with evidence that these numbers had significantly changed, and the influence of native speakers was contributing to grading concerns, we would take a closer look. Additionally, the term ‘native speakers’ may cover a broad range of experience - while one student might be bilingual, another may benefit from having, for example, a French grandparent. It would not be appropriate to consider those two students in the same way.
Comment by Elizabeth Brazier posted on
Thank you for your reply. I am puzzled about how you are able to identify native speakers or even those who are brought up as bilingual from other students as it is not recorded on the exam entry details.
Comment by katekeating posted on
You’re right that exam boards do not collect data that might identify native speakers, or those with any native advantage. Our report describes how we collected this data, but we concluded that it was not something that could be easily operationalised. It was time consuming for schools and colleges, not all of whom responded, and in some cases the responses from students did not align with the information provided by their teachers, which meant we had less confidence in the overall quality of the data.
Comment by Andrea O’Neill posted on
The influence of native speakers at GCSE level will surely be higher than at A level given the smaller numbers taking A level languages anyway. As languages teachers contributing to this blog, we are fully aware of the high number of natives taking the exams and the grades they achieve compared to non natives. It would be useful for you to see what the uptake of MFL at universities is of natives compared to non natives. In addition to the impact that natives have on grading, the punitive marking at both GCSE and A level must be addressed as a matter of urgency if MFL are to be viable subjects in the future. National statistics have shown that students who take mfl at GCSE/ A level achieve a grade lower than in other subjects. Statistics show that THE SAME students who take a language and humanities at GCSE achieve at least one grade lower in their language on average than they do in the humanities subjects, across all ability ranges. This topic will not go away until all subjects are brought in line. It is becoming more difficult to advise students to take a language at GCSE/A level with the knowledge that they will likely achieve a lower grade than other subjects.
Comment by katekeating posted on
Thanks for your message Andrea. Subject experts have raised similar issues with us in the past. Because of this, we completed a comprehensive programme of research into inter-subject comparability in MFL A levels spanning the past 3 years. We spoke to the broadest possible range of interested parties, including higher education, subject experts, teachers and students. We published the outcome of this work, and our decision not to adjust grading standards in these A level subjects, in November. However, we recognise that perceived grading severity undermines confidence. Therefore, we have committed to working with the exam boards to ensure that these subjects do not become statistically more severely graded in the future. And we have confirmed we will now look at the evidence for a potential adjustment to grading standards in GCSE French, German and Spanish. As part of this work, we will consider the level of student performance in MFL subjects at key grades, to address the question of whether the performance at an adjusted (more lenient) grade 4 would still be acceptable.
Comment by Paul Smith posted on
Whilst it may be the case that there are positive elements to the reformed GCSE, the grave concern that we have as language teachers is that students are looking at the new courses and voting definitively with their feet and opting for other subjects. There are parts of England and Wales where fewer than 30% of students are opting for languages. This is not an argument for "dumbing down", far from it, as I do feel that the new GCSE prepares very well for the new A level, however, we do have a duty as language teachers to ensure that ALL students feel that they are capable of gaining a respectable grade in languages. The listening comprehension papers in particular do not, I feel, allow students to demonstrate what they are able to understand and some of the scenarios in the speaking are, frankly, ridiculous. We cannot afford to leave up to 70% of our students behind as we are currently doing.
Comment by Frances posted on
Regarding your reference to grade adjustment in MFL at GCSE and A Level in order to improve inter-subject comparability, could you clarify what conclusions have been reached so far in your discussions for MFL exams 2019? Thank you.
Comment by Ian Dexter posted on
Many thanks for your comment. We published our decision on the A level inter-subject comparability work that we refer to in the post, in November 2018. In relation to A level MFL, we decided not to adjust the grade standards, but instead we are building a requirement into our rules for summer 2019 that exam boards award slightly above prediction, or provide a compelling case to do otherwise.
We’ve started work to look at concerns that students’ grades in GCSE MFL are often their lowest grades. This work includes a consideration of how the new GCSE assessments have functioned, as well as looking at performance of students in relation to the Common European Framework of Reference (CEFR) for languages, but this isn’t yet finished. We will report on this work later in the year. In the meantime, we will not make any adjustment to GCSE MFL in summer 2019.
Comment by Emma posted on
My daughter will be taking French GCSE this summer. I wish it had been made clearer to us that the subject was more difficult than other GCSEs. French is the only subject she is not on target for and will undoubtedly be her lowest grade. She very much regrets taking the subject and we regret encouraging her to do so. Her younger siblings will most certainly not be taking GCSE languages after this experience.
Comment by Liz Munro posted on
I have done some comparison with other subjects for AQA including traditionally challenging subjects such as Chemistry, Physics, History, Geography etc and the percentage required to get a 9 in these subjects compared with MFL is up to 10% lower (in French in 2018 you had to get 80% to get a 9) this is compared to 70% in the above subjects - this is killing students' enthusiasm and turning them off languages altogether - it's incredibly demoralising for us as teachers and will not encourage MFL teachers to stay in the profession - I support the new qualifications but not the excessively harsh grading. Very disappointing and demoralising.
Comment by Kate Keating posted on
Hi Liz. We published in November the outcome of an extensive research programme into the comparability of modern foreign language (and science) subjects at A level, which considered a broad range of sources of evidence. This included a comprehensive programme of research and engagement with the broadest possible range of interested parties, including Ofqual’s Standards Advisory Group, higher education, subject experts, teachers and students. We decided not to adjust standards in A level French. German and Spanish (and biology, chemistry and physics) as we judged there was not a sufficiently compelling case to do so. However, we recognise that perceived grading severity undermines confidence and we have committed to working with the exam boards to ensure that these subjects do not become statistically more severely graded in the future – and full details of the work we undertook are <a href="https://www.gov.uk/government/news/reviewing-inter-subject-comparability">here</a>. We are currently conducting a comprehensive review of grading standards in GCSE French, German and Spanish and we expect to report on this work in the autumn.
Comment by Alison R posted on
As a parent of a young adult taking A Level Spanish this year I have to say that I am very concerned about the comments made by Ofqual about their not being many native speakers taking MFL. Out of the 10 pupils taking MFL in my daughters college at least 3 (30%) are native speakers. My daughter has been struggling to make a decent grade in Spanish despite getting a A in Spanish at GCSE and we are now concerned that because the level for marking Spanish is so high she will not get a good enough grade to be able to take up her place at university (where she will no longer be pursuing a MFL). Surely native speakers have an unfair advantage over non native speakers? Therefore would it not be fairer for their papers to be marked differently to even out the field. I have to say that because of the situation with our daughter and the struggle MFL at A Level has been for her we will be ensuring that our younger son steers completely clear of MFL and have even requested that he be exempt from having to take Spanish at GCSE level as we feel its a complete wast of his time especially as he is unlikely to get a good grade in it.
Comment by Kate Keating posted on
Thanks for your message Alison. In 2017 we conducted a thorough review of this issue. We found native speakers in A level MFL were having a small, yet important effect. We quantified the impact we judged they were having and made an adjustment to grade standards that was then built into future awards
Comment by Paul Smith posted on
Good morning,
It is interesting to note in the British Council Language Trends survey, that many of the concerns making headlines today are those which have been highlighted over many, many years. It is all very well, Ofqual, for you to say that you are "conducting research" to judge comparability between subjects; as teachers we are seeing students being put off MFL because they perceive them to be harder. Until you actually start doing something you will be overseeing the fatal decline of MFL into the status of Latin and Greek, the privilege of the privately educated and a minority subject.
Comment by Matthew King posted on
Having taught A level German for over 25 years, I am sad to see it disappear from our schools. I struggle to understand how OFQUAL cannot see the issue relating to native speakers. The native speaker questionnaires were sent out as the exam season was just starting a couple of years ago, so I am not surprised that the number of responses was low. Exam board markers can invariably tell from the speaking test when they are dealing with a native speaker - this would be a quick and simple way to way to measure the numbers involved. The new A level courses are interesting and engaging - however, I feel there is a moral obligation to let students know that they are not dealing with a level playing field if they opt for A level German.
Comment by Ian Dexter posted on
We conducted research into the potential effects of native speakers on standards in A level MFL ahead of the summer 2017 awards. The research suggested that there is likely to be a small, yet important effect, of native speakers in A level MFL. We asked exam boards to take action during the summer 2017 awards to reflect the findings of the research and made an adjustment to grade standards that was then built into future awards.
The research highlighted the challenges of identifying native speakers. Whilst this might seem obvious at the extremes – a student that was born and lived in Germany until age 16 would likely be a native German speaker, and a student who had no exposure to German outside of their A level would likely be a non-native German speaker – there is a grey area between. This could include students who might not conventionally be thought of as native speakers, but who have had exposure to a language they were studying beyond their experience at A level. Essentially, native speakers are likely to exist on a continuum. We therefore do not think that identifying native speakers is straightforward, and do not think that asking exam boards to monitor this each exam series is feasible or indeed proportionate.