Fostering assessment for student mobility

COVID19 tore up our rule book. In this crazy new world we have to acknowledge that the days of long lines of students, sitting their examinations in halls, with moderators walking the aisle are numbered. Pen and paper assessment has already been viewed as archaic.

Fortunately technological advances have brought transformational and momentous change in the online education and assessment experience.

Increasing numbers of universities are moving fully online by summer 2021 owing to the Corona Virus pandemic, (ref. Guardian: Oxbridge to replace summer exams with online assessments).

The move to online delivery and e-assessment delivers both opportunity and fresh challenges in assessment practices rather than traditional methods. One area that is clearly challenging is the support of learners who require additional assistance. Beyond the lecture theatre and the seminar room, technology can and must offer educators the ability to continue support and collaboration with learners, assessors and awarding bodies.

The move to online modes of delivery are being positively received by both staff and students. However, issues for the accessibility of online assessment remain a challenge. The resulting high levels of concurrent traffic on examination systems, combined with a new and unfamiliar (to students and staff) assessment experience, means institutions will encounter previously unseen issues arising during peak exam periods, which will require careful management and support.

Catalyst has worked with several institutions globally to provide a ready-made e-assessment solution built on the world leading open source learning management system Moodle. We currently work with Monash University in Australia, Dublin City University in Ireland and Concordia University in Canada.

DCU e-assessent solution

DCU delivered the e-assessment solution in 2020, 1,200 concurrent examinations were achieved on the busiest days. The model used was designed to support a potential concurrent load of 4,000 students taking an exam at any one time. Both open book assignment upload and quiz type multiple choice question methods were used. In addition to ensuring students were able to continue to complete assessment towards their degree programmes, staff were able to benefit from significant efficiencies in the grading and feedback flows. Where multiple choice question approaches were utilised, grading and feedback is automated in the most part. Where assignment upload methods were utilised, grading, feedback and mark submission all take place within the Moodle grading interfaces – reducing the historic methods of shipping piles of exam papers around.

Concordia University chose to switch to the e-assessment solution with Catalyst for the majority of final, summative and midterm exams during the summer and winter 2020 terms This was administered through COLE (Concordia OnLine Exams). By Oct 2020, 25,000 high stakes exam sittings were delivered through the COLE. Supporting thousands of student candidates every week.

“We’re impressed with its user-friendly design that addresses the needs of students and faculty.” Sandra Gabriele, Vice-Provost of Innovation in Teaching and Learning.

AWS cloud tech hosting

The e-Assessment platform is built on a dedicated instance of the Moodle LMS application, with exam-specific features and customisations – hosted on Amazon Web Services (AWS).

AWS enables exceptional levels of performance and assurance, when required, at peak assessment periods. The scalability of hosting on AWS Cloud ensures a robust and performant LMS giving peace of mind for educators and students alike. The financial advantage of only paying additional power throughout exam periods, instead of investing heavily in more on premise IT kit for the short exam periods is a win win situation.

E-Assessment solution for Monash
Before Covid-19 hit Monash University, supported by Catalyst, embarked upon their e-assessment project to replace paper based examinations with a fully online digital solution.

Whilst this solution was then released for wider adoption globally in the higher education space, when Covid-19 did arise, this was a ready made solution to the very real and pressing problem.

Catalyst and Monash found that having an exam platform separate from the LMS that is used for teaching, ensures a clean division and simpler user experience. Students access e-Assessments from a clean and dedicated instance away from the core LMS instance. Our e-Assessment service can consequently work alongside any LMS that might be in use in an institution.

At Monash the e-Assessment solution is a secure and robust platform, employing a user friendly interface designed for students by students. They use a locked-down browser which restricts students from accessing unauthorised resources during assessments. It has been built to be inclusive and authentic to enable students to demonstrate evidence of learning at the appropriate level of study.

Spending significant time with educators, academics and learning designers particularly during exam pilots, Catalyst has added some key capabilities and customisations to streamline the overall exam experience delivering on expectations of staff and students.
 

What does e-assessment look like to the student?

Students can take notes, highlight text and have a full screen work area to write their answers. They are able to see which questions they have and have not answered. There are progress indicators and question flags for questions that might merit a review if time permits. All answers within an exam can be reviewed and changed before the final exam submission. Exam clocks monitor time usage and are visible to the students in the way wall clocks were in physical paper-based exam scenarios. There are increased notation spaces for highlighting and marking up work. To ensure academic integrity, 15 second snapshots are taken to observe students progress, in real time. Where accessibility and disability have an additional time requirement, educators are given permissions to manually adjust the time allowances.

Benefits of e-assessment to educators

There are numerous benefits of e-assessment. The difficulty of reading illegible hand-writing is eradicated. There are increased time-saving opportunities facilitated by auto-marking. Examination papers not lost and no wait is incurred for paper batching. The flexibility to access anytime and mark students exams from anytime anywhere is a clear advantage. Within the assessment solution for continuous assessment there are enhanced insights into the progress of students using the analytics capability particularly vital for continuous assessment.
Many universities have horror stories of losing completed exam papers, and this method completely removes those risks and issues.

Academic integrity is a very big point of discussion in this space, and no single agreed position has been reached according to all discussions Catalyst have shared with our partners. If proctoring is required, we are able to facilitate integration to a number of the leading and well known proctoring solutions.
The right remote proctoring/invigilation solutions should be secure, minimally invasive to the exam taker, and easy-to-use for educators – but this should be considered on an almost case by case and requirement by requirement level.