<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=353110511707231&amp;ev=PageView&amp;noscript=1">

Can Online Proctoring and Test Center Proctoring Live in Harmony?

December 2, 2020

The balance of online and test center proctoring has certainly shifted in the past year with the challenges of the pandemic and regional lockdowns. As a result, online proctoring has seen accelerated implementation as credentialing bodies attempt to continue their testing streams in the face of marketplace and environmental challenges.

The shift away from reliance on test center delivery has necessitated critical changes that needed to be implemented very quickly to incorporate online proctoringAt PSI, this included changes to: 

  1. Policies and Procedures – Certifying bodies identified new proctoring modalities and implemented PSI Bridge to augment the test delivery  process. Consequently, proctor administration rules and exam security concerns had to be addressed in the context of this new environment. 

  2. Vendor Relationships – Weekly implementation meetings for online proctoring, as well as monitoring statistical outcomes, were introduced with no changes to COI or NDA agreements. 

  3. Technology Configurations – There was a move to a new technology-enabled scheduling system, the implementation of single sign-on, and the development of new APIs to transmit candidate data 

v4-infographic-1-1In addition to these and numerous other process changes, communication to candidates was another critical area that required revisiting. For example, during lockdowns that closed test centers, emails had to be sent to candidates who were already registered for exams so that extensions for eligibility could be provided. Communications about computer specifications for online proctoring were also required, as well as clear instructions on accessing support in the event of technical issues.  

Testing organizations have had to address the impact of technical issues on the candidate experience, particularly when those issues prevent candidates from completing their test. When this happens, implementing processes that preserve the continuity of the candidate journey has needed careful consideration. Issues such as the extension of eligibility or the provision of timely re-take opportunities, potentially under different delivery modalities, must be clearly defined and communicated. 

Download the Full Infographic: Evidence Supporting Multi-Modal Testing in 2020 

Furthermoreprivacy policies and security measures had to be evaluated and revised to take account of the fact that those taking exams via online proctoring may have utilized any the following: 

  • Lockdown browsers, limiting usage of other apps, tabs, messaging, sites, etc. 

  • Microphone/webcam check 

  • Candidate authentication using valid ID 

  • Camera panning and room scans 

  • Recorded exam sessions 

  • Exported data transfers 

  • Proctor troubleshooting 

With the rapid shift to online proctoring, credentialing organizations were also faced with determining and deciding between what constitutes a major infraction versus a minor infraction. This means that the role of the proctor in online proctoring has become even more important.  

Trained third-party proctors, like those at PSI, have no vested interest in the certification program or the candidate taking the exam. Proctors participate in live monitoring with a set of predetermined rules set out by the credentialing organization to help detect any violations, thereby giving a consistent experience across all test takers. Chat and audio capabilities allow PSI proctors to communicate with test takers if needed as they are taking the exam, with a proctor incident log to document the test taking experience across candidates. 

As the market moves into phased re-openings, we are seeing test centers re-opening and both modalities (in-person and online proctoring) becoming available to test takers. Studies show that pass rates in online proctoring are similar to in-person pass rates. Performance indicators are similar overall regardless of the testing modality used. 

With candidate experience surveys, credentialing organizations can further improve their online proctoring processes and address any concerns before they become larger issues for test takers. Survey responses overall have indicated satisfaction with the remote scheduling and registration process as well as the experience itself. 

Online proctoring, like any technology, is on an arc, it continues to improve as more organizations utilize it and refine their processes. In addition to general improvements, online proctoring is seeing some future-forward implementations such as gaze detection, facial recognition, mobile device auto detection, and automatic candidate motion alerts, among others. The technology continues to mature in an effort to accommodate test takers requiring the convenience of online proctoring.  

As we move through the challenges of a pandemic and its recovery, it is clear that online proctoring has and will become a consistent and more pronounced presence for credentialing organizations looking to keep their businesses moving. And that is not to say that test centers will become obsolete – but these modalities will likely easily co-exist with no marked difference in metrics or performance.  

Statistical analyses suggest that outcomes in either proctoring modality do not differ, and candidate satisfaction appears to be similar in most respects. Having both (with the assumption that it is safe to continue test center operations) also gives candidates the opportunity and flexibility to choose which approach works best for them. However, as with any change in test administration, work must be done to update policies, procedures, and candidate communications to ensure a seamless and secure testing experience. 

Multi-Modal Testing Webinar 2020 and Beyond

Dr. Amin Saiar Dr. Amin Saiar is the Vice President, Psychometrics at PSI Services and joined PSI in January 2013. During his 12+ years of experience in the assessment industry, he has conducted job analyses, psychometric evaluation of credentialing programs, accreditation consulting, examination development, and standard setting for numerous certification and licensure programs. He is experienced in a variety of assessment formats including written examinations with alternative item types and performance-based examinations as well as assessment applications outside of credentialing including medicine, public health, licensure, career development, and organizational psychology. Dr. Saiar served as a commissioner for the National Commission for Certifying Agencies (NCCA) for 6 years, was a chapter author for the Third-Edition ICE Handbook, and is a current member of the Association of Test Publishers (ATP) Certification and Licensure Division leadership. He received his BA in Psychology from the University of California, Los Angeles and his PhD in Industrial/Organizational Psychology from the California School of Professional Psychology.