One of the many potential benefits of e-assessment is improved accessibility for learners with disabilities: technology can help reduce or remove some of the barriers that learners can face. However, many e-assessments allow relatively limited adjustment for learners with access requirements, and often assessments are not optimised for easy use even for those without special access requirements.  For technology to deliver on the promise of improved accessibility, it is crucial that this accessibility is considered at every stage of the assessment development process. In AlphaPlus’ current work with the Welsh Government on the Welsh national personalised assessments, we have made sure that the assessments have been designed with learners’ access in mind.

Background

In Wales, all children aged 7 to 14 (years 2 – 9) take national assessments in procedural numeracy, reading and numerical reasoning. These assessments are designed to identify areas where children and teachers need to focus their efforts in order to make progress.

Welsh Government made the decision to move these previously linear, paper-based assessments online, taking advantage of the benefits of e-assessment. AlphaPlus is leading a multi-partner project team on behalf of Welsh Government to develop these assessments.

Since the project outset, accessibility and inclusivity have been key requirements of the new personalised assessments, as every child in Wales who is cognitively able should be able to take the assessment on screen.

The numeracy (procedural) assessments are now live in schools. The reading assessments (in Welsh and English languages) will be available on-screen during Autumn 2019. The numeracy (reasoning) assessments are being developed, ready for their introduction during Autumn 2020.

Once these national tests move online, absolutely all the assessments are delivered on-screen – there are no paper-versions, no large-font, no completely braille printed tests (although we do have some braille booklets to support the onscreen assessments), etc.  Every child that is cognitively able to take the assessment, takes it onscreen, with access support, either in the form of technological support or by an adult, where needed.

What we did

In order to enshrine accessibility within the assessments, we adopted a universal design approach, building accessibility into all technology and content development from the outset. The Welsh assessments are WCAG2.0 AA compliant and accredited by the Digital Accessibility Centre. We placed accessibility experts within the software development teams and work closely with Pia, an accessible assessment materials production company, in preparing assessment content. We consulted with the Welsh Association of Vision Impairment Educators (WAVIE) and trialled the modified versions of the assessments with learners with visual impairments during the development process.

General best practice for accessibility

As well as considering best practice for accessible e-assessment, the assessments were developed with general best practice for accessibility in mind, for example:

• All items in the procedural assessments have been written in the active voice, using short, straightforward sentences where appropriate.
• Where possible, we have ensured that the language used is accessible to learners e.g. avoiding difficult concise forms of irregular verbs in Welsh, and ensuring that the impersonal form of the verb is not overused in texts.
• Diagrams, illustrations and photographs have been chosen carefully to support the question text.
• Names and images have been chosen to reflect the diversity of society in the twenty-first century.

Adaptive

The assessments are adaptive – this means that as the assessment progresses on an individual basis for each learner, the questions presented are selected based on the learner’s preceding responses.  If they get questions correct, the questions get harder, and vice versa.  This has two key impacts, both of which offer improvements to accessibility (as well as other educational benefits):

1. The assessments are engaging for all – questions are usually at or about the right level of difficulty for learners and provide an appropriate level of challenge. Paper tests, which are the same for all learners in a given school year, typically have significant floor and ceiling effects (too easy for some learners, too difficult for others) in order to cover the range of abilities within a school year.
2. The adaptive assessment typically takes less time than an equivalent linear test to reach an equivalent measurement accuracy. Shorter tests mean better learner engagement.

Consulting experts and stakeholders

Throughout the project we have worked with stakeholder groups who work with learners with disabilities, attending meetings and presenting plans to ensure stakeholders are on-board.

We worked closely alongside the Modified Test Agency (Pia), commissioning their experts to review all items before they were finalised. This brought another layer of quality assurance, ensuring that items are as accessible as possible to the majority of learners. They also advised what adaptations should be made for specific groups of learners, and how.

When preparing specifications, we included discussions with modification experts who commented on elements that would be difficult to modify later on, for example, when selecting reading texts for a comprehension test, they highlighted highly visual content which may not have been accessible to visually impaired learners. When requesting technical developments, e.g. new item types in the software, the requests are reviewed by accessibility experts within the technology teams.

During the initial planning stage, we conducted surveys with special schools and representative bodies to create a snapshot of the range of disabilities to be accommodated and of favoured tools and approaches in use. We also ensured that special schools were included during the trialling phase, working with learners with various physical and behavioural issues to ensure the best possible access.

Platform

The national personalised assessments use the Surpass: Assessment Platform by BTL, a platform with years of user experience refinement for many of the world’s largest e-testing providers, including Prometric, AQA, WJEC and City & Guilds.

Surpass allows tests to be provided on Windows and Apple desktop PCs and laptops, iPads, Kindles and Android tablets, ensuring that learners can access the test through a device with which they are comfortable.

Surpass provides the following accessibility support for learners in test delivery:

• Colour preference options for dyslexic and visually impaired users
• Alternative text descriptions for images
• Additional time for learners with special requirements
• Keyboard only navigation
• Text-to-speech support through JAWS screen reader
• Screen magnification support.

Extra materials

In administration materials, we include detailed sections on how materials can be accessed by different groups, for example those requiring magnified versions or colour overlays.

When preparing familiarisation materials, we have developed materials that reflect accessible versions of live tests.

A small number of the questions in the National Numeracy Personalised Assessment (Procedural) require the use of tactile diagrams for learners with visual impairment. It has been necessary to include these to ensure curriculum coverage for learners who are using screen readers or having the assessments read by their teacher or other adult. The learner will only need to use the booklet if one of the questions requiring a diagram is selected by the system.

The reading texts and questions are both provided in Braille format so the learners can access them in the format they would usually access text.  The answers are input onscreen so that the benefits of personalisation of selected texts and questions, detailed feedback and immediate feedback are available to learners who require the Braille versions.

Modified tests are available to support learners with visual impairment or learners who normally use large print as part of their normal classroom practice. There is also guidance to support those administering the modified assessments to learners with a hearing impairment or who use sign language.

Timing and administration

The personalised assessments do not have a fixed duration, although there is a maximum number of questions and assessments are generally expected to take been 20 and 40 minutes for procedural numeracy and between 35 and 45 minutes for reading. The system stops the assessment when it has made a reliable decision about the learner’s ability. Therefore, learners can take as long as they need to finish tests, without feeling the pressure of an imposed time constraint.

Some learners may have difficulty concentrating or experience fatigue due to a health problem or a difficulty in processing information. If it is considered that the learner requires a break to allow them to perform to the best of their ability, this can be done by pausing the assessment in the assessment website. The learner can log out and then log back in to resume the assessment at the point they left it, although the assessment must be completed on a single day.

Schools can choose to assess classes, small groups or individual learners according to their facilities and at a time that works for them and their learners. As the personalised assessments are different for each learner, there is no need for a whole class of learners to take the assessments at the same time or for concerns about cheating.

Conclusion

From the project onset, the Welsh national personalised assessments have been designed to allow learners with different accessibility needs to fully demonstrate what they know and can do. Where accessible versions are needed, we have worked with key stakeholders to devise assessments that meet the widest possible range of access needs.  We have also worked with a wide range of commonly used access technologies and access arrangements. For many learners, the use of e-assessment will reduce barriers present in the previous paper-based tests. By moving these national assessments to a personalised, on-screen format, Welsh Government have taken full advantage of the benefits of e-assessment and allowed learners with a wide range of accessibility needs to access the assessments.