You may recall that we launched a research programme looking at the four new Ofqual accredited GCSE maths specifications in late January. If you need a refresher on what we’re trying to achieve you could watch my earlier video or read Glenys Stacey’s blog from last month. At its launch we committed to complete our work by the end of April, so we’re now roughly halfway through, and I’m happy to report that we remain on track. But let me provide a little more detail on progress across each of the strands.
The first strand is assessing the relative mathematical demand of questions in the boards’ sample assessment materials against those from current GCSE maths papers and from 12 international jurisdictions. Around 40 PhD maths students have been looking at pairs of questions and been answering for each pair: “Which question is the most mathematically difficult to answer fully?” We received their responses at the end of February, and we’re now in the process of analysing the data.
Strand two has involved the large-scale testing of the boards’ sample assessment materials on current Year 11 pupils. Some 4,000 students drawn from across the country each sat one full question paper under exam conditions during the three weeks to 13 March. The exam board whose paper each student sat was drawn at random – each board has produced foundation and higher tier papers so the school could nominate at the outset which tier each student sat, but other than that it was random. A team of examiners has begun marking the papers this week and the results will be available to us in early April.
In our last update we said we had decided to change tack in strand three as a pilot of our original approach had suggested we were unlikely to obtain meaningful data. Consistent with the revised approach, we asked around 50 Year 11 pupils to answer a range of problem solving questions in early March. The next step is to ask a number of teachers to identify which of the questions have allowed the students to demonstrate their problem solving skills to the greatest extent.
So that’s the work we had planned, but we've decided to take one further small step. We will be presenting 5 maths experts with around 30 problem solving questions drawn from across the four exam boards' sample papers and asking them to judge where differences and similarities between the questions exist and on what basis. Combinations of questions will be repeatedly presented in groups of three to the experts. We will be asking them to pair two questions together based on similarities and leaving one separate. They'll then be asked for a reason for their decision. This will be repeated until the reasons being given for the differences/similarities dry up. Coming out of this exercise will be a number of different scales that the questions will then be scored on with the scores providing information on any systematic differences across the questions. This is a relatively swift exercise, taking only a few days, but it should provide useful additional background to our other work.
A more detailed summary of all our research strands can be found on our website. I would finally like to say a big thank-you to the many pupils, teachers, PhD students and maths experts that are supporting our research. Their engagement and assistance has ensured that we are progressing very smoothly towards our goal. There is still more to do, and we will continue to provide updates over coming weeks.
Dr Ian Stockford
Acting Executive Director for General Qualifications, Ofqual