Dave Mellor, Deputy Director of Assessment at AlphaPlus, shares his thoughts on the summer 2021 GCSE and A-level exam session.

Over the last few months Ofqual have published their decisions on changes to the 2021 GCSE and A-level exam series to try and deal with the impact of Covid-19 on students’ education.  The changes are relatively modest and focus on introducing changes to accommodate social distancing in some subjects and to allow some optionality for teachers and students on topics to teach and learn for other subjects.  They have also changed the exam timetable so that most of the exams are after the May half term, with just GCSE English and Maths before, in order to give more teaching time for students.  The results day for A-levels has also been pushed back a few days.  The overall aim is to try and reduce the burden on schools and students as they try to fill any gaps created by the closure of schools and the move to remote teaching and learning. 

There have been a number of commentators who have said that the changes agreed are too small and do not reflect the needs of the students.  Whilst making changes to what is taught and assessed sounds easy, the reality is that there are real challenges in making changes to an assessment system in mid-flight, including changes to the exam timetable.  In this blog I want to look at why it is so difficult to make changes to the exam timetable by looking at the process for managing the assessments.

Moving the exam timetable back a couple of weeks sounds like it should be fairly straightforward but the process by which results are created is both labour intensive and complex.  The volumes of students taking the exams are also very large and the stakes for students are high so cutting corners to try and speed the process up would risk incorrect results.  I will illustrate the process used for moving from an exam paper to results using GCSE English Language as an example. 

Approximately 525,000 students each year take GCSE English Language with the most popular (for this subject) awarding body.  This means that for each paper, this exam board needs approximately 2000 examiners to mark and check the marking of other examiners.  Most of these examiners are current subject teachers and they do the marking in evenings and at weekends whilst doing their teaching jobs.  This means that a minimum of three weeks is allowed for the examiners to mark their allocation of scripts.

The first stage in the process is for the schools to gather the students’ scripts together after the exam and post them to a scanning bureau.  The scanning bureau scan the student scripts to create electronic images which can then be made available to the examiners.  Electronic marking from scanned image is done to allow real time monitoring of the marking quality and retrain or stop examiners who are not applying the marking standard correctly.  With delays in posting materials and the time taken to scan the scripts, it is usually a couple of days after the exam is sat before the first images are available to mark. Scanning over half a million scripts is not a trivial process (and this is for just one subject for one board) so the time taken to do this, even with state-of-the-art technology, is significant.  Most bureaus work 24/7 during the exam series to complete scanning and ensure a steady supply of scripts for examiners to mark. 

Before the examiners can begin marking, they have to be trained on how to apply the mark scheme.  This means a group of senior examiners have to read and review a selection of exam answers to identify suitable training materials.  They also need to check that students have responded to the questions as expected. If unusual but correct responses are identified that have not been included in the mark scheme, the mark scheme will need amending.  Reviewing sufficient scripts, amending the mark scheme and then producing the training materials usually takes a few days.  The training materials are then made available to the examiners online and they work through these and mark a sample of assessments.  This sample of assessments is then reviewed by a senior examiner for the examiners in their team to check that the mark scheme is being applied correctly.  If it isn’t additional training may be provided or the examiner may be stopped from marking.   

Once the examiner is approved for marking, they begin marking the scripts and the quality of the marking is reviewed.  If an examiner is not marking to the right standard they will be stopped and either retrained or their marking contract will be terminated.  The marking should take three weeks, though a significant number of examiners do not manage to mark all their scripts by the final deadline because of unexpected personal circumstances. 

Inevitably there will be a number of examiners who are stopped from marking or who drop out for personal reasons or who do not complete all their marking.  This means at the end of the initial marking period the unmarked scripts will be marked by examiners who are willing to carry on marking.  It is often then a couple of weeks or more before all or nearly all scripts are marked. 

Once marking is nearly complete the statistical analysis of the exam takes place and an awarding meeting occurs.  The meeting of senior examiners reviews the statistical evidence and a selection of student answers to determine where the grade boundaries should be.  These meetings are supported by statistical experts and usually last a couple of days for most subjects, though those with lots of papers can take longer.  In any single awarding body more than 100 awarding meetings have to be scheduled and supported by the statistics experts so they usually last for about 4 weeks and include weekend working.

Only once the grade boundaries are in and the marks submitted can the students’ grades be calculated.  There are then checks on these to make sure that they are correct before the results are then ready to be shared with schools and colleges, UCAS etc.

The exams are scheduled between the middle of May and the end of June and results are issued in the second and third weeks of August for A-level and GCSE respectively.  This gives between 7 and 12 weeks for all the work to be completed depending on when in the exam series the paper is sat.

If the exam series is delayed, then there are significant risks that results cannot be issued on time.  The markers would struggle to complete the marking in a shorter period of time as they are still teaching. The awarding process cannot be condensed significantly as the meetings already run over the weekends and there aren’t sufficient people with the right expertise to manage more meetings on the same day.  If the exam timetable were to start later and end as they do now so results can be issued on time then this would also generate a lot of clashes for students where they have more than one exam on at the same time which the schools would have to manage.

Even a high-level description of the process of taking a script and issuing results show that the multiple complex processes overlap with each other as the papers are sat by the students.  Coordinating and delivering this is a significant piece of work.  For the largest of the English Language awarding bodies it involves over a thousand staff, hundreds of temporary staff and 25,000 examiners.

Ofqual had a difficult choice to make.  Any move to the timetable to free up teaching time creates a risk that not all results will be issued on time if the results days remain the same.  If the results days move, then it risks delaying the start of FE and HE courses which then has a knock-on impact on students and FE and HE institutions.  They have worked with the board to delay the start of most exams and push the A-level results day back a few days as a compromise solution.

Conclusion

There is no easy solution to cope with the impact of COVID-19 on the students for summer 2021.  Ofqual have made the changes that they believe are plausible whilst maintaining the validity of the exams and managing the risks to delivery of accurate results on time. 

The use of comparable outcomes to determine grade boundaries year on year will mean that if the whole cohort performance in a subject is, on average, worse this year than previous years then the mark needed for a particular grade in 2021 will reduce in order to ensure a similar national grade distribution for the subject. This will not compensate those students who have suffered a relatively greater loss of teaching because of COVID-19 restrictions, but then the exam system was never designed to take into account the differences in the teaching and learning experience of different students in different schools.

The assessment system can only deal with the national picture and cannot make allowances for different quality of teaching experience – it can only assess what the students know and can do at the end of the academic year.  It is up to others in the education system to try to manage the impact on students and reduce as much as possible any unfairness caused by the disruption.