Covid-19: what we’ve learnt running trials in a pandemic

It has been almost 8 months since schools first shut to limit the spread of coronavirus. As well as all of the obvious disruptions, Covid-19 has had a massive impact on our trialling work, delaying or otherwise changing many of the trialling projects we had in the pipeline. As part of our work developing assessments, we undertake a lot of trialling – learners trying out assessments so we can calibrate the assessment content, and obtain feedback from learners and teachers about how the tests work. In the last three years or so, we’ve delivered 25 major trials (and many smaller ones) with over 135,000 learners, and 11 residential marking operations with over 250,000 scripts marked, with more than 75 test administrators and markers.

Now, schools have been open for two months. We’re still running trials, despite the previous local restrictions and the new national lockdown, currently four major trials for over 20,000 students in schools. All are non-statutory so require voluntary participation. Despite obvious challenges, we are on track to recruit the required numbers. Here are some of things we’ve learnt over the past few months.

Schools are still keen to take part in trials.

This is the most important factor, as it was one of the greatest unknowns before we resumed our trials service. Initially we were concerned that schools would – understandably – be too busy to participate in trials after months of pupil absence.

However we have instead found that schools want to learn how their students are doing after such a long time away from lessons, and so many want to take advantage of free tests with feedback on how their pupils are performing.

This isn’t just limited to our trialling projects. In Wales, almost 20,000 national adaptive personalised assessments were scheduled in the first four weeks of term alone, as schools wanted to take advantage of the formative benefits of the assessments.

As always, and quite reasonably, schools require that the trials are as low hassle for them as they can be. If there’s technology involved (onscreen tests, for example) then it has to work, the test content has to be properly organised and the operational arrangements must be clear. Quick and helpful support over the phone has to be available on tap, too.

However, schools can’t guarantee that they won’t be forced to drop out.

With whole classes needing to isolate if exposed to the virus, schools are unable to guarantee that they will be able to take part in trials. This is completely understandable, and our role involves doing everything we can to support schools to make taking part easier. This includes, for example, extending the length of the trialling window to increase the likelihood of schools participating, as well as overrecruiting to mitigate the impact if a school is forced to drop out.

Schools have concerns.

As well as their usual concerns (how much time the trial will take, whether it will be too large a distraction from teaching, what support they will receive, what feedback they will get from the assessments, etc.), schools want to know how COVID factors will impact the trial. We can help with these concerns by providing additional advice and support.

We can’t send our test administrators into schools.

In the past, we used to often send a trained test administrator into schools to ensure that everything ran smoothly – this reduced the administrative burden on the school and also meant we could ensure the trial was delivered precisely according to the protocol. Even before this new national lockdown, we were reluctant to send in test administrators to schools (and many schools don’t permit access to the school for non-staff particularly where the purpose is non-essential. This has meant that we needed a new strategy for trials that would have previously been administered by one of our trained administrators. Trials are now either being taken online, or we are sending test papers into schools to be administered by teachers and returned to us. This leads on to the next point…

Trials need more project management and more time spent speaking to schools.

If assessments are being taken online, we need to walk teachers through the assessment system, and be available throughout the testing window to answer any technical questions that arise.

If schools are administering paper tests themselves, we need to make sure that they understand how to process works, and answer any of their questions. We also need to arrange the drop off and collection of the paper tests. When our test administrators are going into schools, they take the papers with them. Arranging for papers to be collected and dropped off with administrators is much simpler.

Despite all this, our trialling projects are going well.

Our trialling team has responded incredibly well to the new challenges, and have successfully recruited for four non-statutory trials with a combined total of over 20,000 learners. This is all down to the flexibility and decency of the schools we work with (thank you to all of them) and to our inhouse team who are knowledgeable and committed to making trials a success, not just for our clients but also for the school communities we work with.

Find out more about AlphaPlus’ trialling capabilities here.