What We're Most Excited for at ATP 2019

Association of Test Publishers (ATP) Conference 2019 kicks off next week, and in addition to reconnecting with friends, colleagues, and clients, we’re looking forward to sharing ideas and discussing the latest innovations in testing. Among so many important topics, we’re excited to discuss, share our ideas, and hear other’s opinions and experiences regarding the following:  

New Methods in Test Security  

Today, it’s not a question of whether a test taker will try to steal your content, but when. In an era where cameras can be in an examinee’s contact lens, PSI has been using data forensics, facial recognition, and Artificial Intelligence (AI) to help identify these test takers even when traditional security precautions cannot; We’re looking forward to sharing how we’ve challenged the status quo of testing security and evolved its technology and products for today and tomorrow. 

Online Proctoring 

As we watch remote proctoring grow, the future of remote programs has evolving security needs. By actively piloting the role of AI and machine learning in proctoring in a deliberate and research-driven way, we’re learning how this will give testing organizations increased confidence in adopting this delivery approach.  

Showing a critical evaluation path for whether programs are ready for a remote proctoring delivery method is a key consideration, so we’re looking forward to hearing David Foster, Liberty Munson, and Manny Straehle’s session that digs into this question. 

We’re also excited about Ben Taylor’s presentation on the past, present, and future with AI. This is geared up to be an awesome exploration of the true possibilities of AI, not just the hype or horror stories that we read about in the press. 

Standard-Setting
At PSI, we're often asked what the “correct” standard is. People want to know about the percent correct or the pass rate, but the “correct” standard is the one that supports the mission of the organization and is consistent with the intended meaning of the credential. We’re looking forward to our panel on issues related to standard-setting studies from the perspective of a psychometrician, a participant, and an organization’s decision maker. 

Recertification 

Recertification is getting more attention. We can work together as an industry to get closer to a shared understanding of its purpose, develop best practices, and help inform the public’s and regulators’ mindset on the topic. 

Maintenance of certification is so varied because it’s complex. Having a better understanding of the different design decisions involved will help ensure that we achieve our objectives through the processes we put in place. This is why we’re excited to explore some key questions at ATP 2019: 

• Should an assessment component be required for recertification?  
• If used, should the recertification exam be the same as the initial certification exam or a different exam? 
• Should continuing education be a requirement for recertification?  
• Should recertification models differ by subgroup characteristics, such as years of experience, specialty area, or work setting?  

We’re also looking forward to hearing the trends and innovative ideas in recertification and renewal, gathered from the results of the ATP Certification and Licensure division subcommittee’s survey of credentialing organizations.  

Assessment Innovation and the Validity-Diversity Dilemma: Challenges and Opportunities 

Organizations that use test scores to make high stakes decisions about people, such as college admissions and employee selection, often face challenges in balancing competing goals. For example, validity in predicting scholarship and effectiveness, utility, and diversity. This is especially true with cognitively-loaded assessments that tend to be the most predictive of important outcomes related to academic and job success. At the same time, these measures by themselves may manifest exclusionary impact for certain demographic groups with respect to test performance, resulting in lower selection rates and less diversity (i.e., adverse impact).  

Validity-diversity dilemma can be addressed by examining recent trends, such as technology-enhanced measurement, increased use of non-cognitive measures, and scoring techniques (e.g., stealth measurement). We look forward to discussing this in more detail at ATP. 

As psychometricians, practitioners, and innovators, we’re excited to not only share our knowledge about scoring, fairness, and the latest methods in test security, but to learn what other top researchers are doing as well. We’re looking forward to lively conversations and interesting sessions at ATP Conference 2019! 

For a schedule of our sessions at ATP, visit psionline.com/atp-2019. Not going to ATP? You can still sign up to receive the content we'll be covering at ATP!