More and more clients are asking us how to improve the candidate testing experience with the use of innovative item types, also known as alternative item types (AITs). Here we explore AITs and some key benefits and considerations to think about before embarking on changes to your program.
What is an Innovative Item Type?
Well, in short, they are items other than traditional multiple-choice items (MCQs). MCQs are characterized by response options following a written prompt, with one keyed (correct) response and dichotomous (right or wrong) scoring. AITs are characterized by changes to the formula in terms of format, display, and/or scoring. Perhaps the most commonly used AITs are:
- Multiple Choice Multiple Response (MCMR) – two or more keys, often more than four options
- Hot Spot – Answer in the form of clicking on part of graphic
- Drag and Drop – Clicking and dragging text or graphics to match, sort, or rank
- Written Response – An open response to a prompt such as an essay or short written answer
- Video Prompt – A video is part of the item stem
Build a More Engaging Candidate Experience
In today’s technology-driven world, we have become accustomed to high-quality visual interfaces. It pays to make sure your candidates get the best experience from your organization and that your organization gives them the best exam experience possible. To this end, it is important to ensure the requisite technical capabilities are in place to realize your testing vision. Here at PSI, our test delivery platforms are built to support a wide variety of item types and associated media like graphics, audio clips, and video. Decisions about formatting and display can have a dramatic effect on the candidate experience and the validity of assessment outcomes. Parshall and Harmes (2008) identify a recommended process for the development of AITs and there are certainly several decision points throughout. The guiding principle is to consider the experience intended for the examinees when developing, testing, and evaluating the type and number of innovative items introduced into your assessment.
Bring Higher Fidelity to Your Assessment
In our day-to-day working lives, we typically are not presented with multiple-choice questions from our colleagues and clients! So how much does your exam content reflect what the actual job looks like? AITs may be used to increase an examination’s fidelity, that is, its ability to more closely reflect actual work activities. That said, higher fidelity does not necessarily equate to higher validity. Validity (that is to say, construct validity) indicates how accurately the assessment reflects the construct (i.e., the thing you’re trying to measure). No major change to a certification program should be made without considering the impact to the validity of assessment outcomes. At PSI, our psychometricians and test developers always look to design examinations with validity in mind to ensure that assessments outcomes are meaningful and defensible.
The image above is an example of a Hot Spot AIT.
About the Author
Dr. Amin Saiar is a Psychometrician and Credentialing Expert with a passion for developing client-centric assessment solutions and communicating complex psychometric concepts in practical terms. He also volunteers as a Commissioner for the National Commission for Certifying Agencies (NCCA) and as the Secretary of the Certification and Licensure Division of the Association of Test Publishers (ATP). Contact Amin if you want to talk about credentialing, psychometrics, structural equation modeling, geography, association football (soccer), or esoteric films.