“Did you get the question about pancreatitis?”
“No – Did you get the question about rhabdomyolysis?”
“What?! You got a question about rhabdomyolysis? That’s such an easy topic – your version of the exam must have been way easier than mine!”
“Yup, I guess I got lucky. Sorry you got the hard version!”
Some variation of the above dialogue has been heard from high school halls to high-stakes exam centers. The notion of different “versions” of the same exam has been a part of examinee discourse for decades.
The problem itself is self-evident: If a certifying body is administering an exam, they will often want more than one version (or “form”) of that exam. The benefit of using multiple forms is also evident, as test security can be greatly increased. A person who does not pass Form A can be given Form B on their next attempt instead of receiving the same Form A on attempt #2. It can also make collusion and answering using advanced knowledge more difficult. Administering multiple concurrent forms can reduce individual item exposure (i.e., the number of instances that any given item is seen decreases).
Related: Using Innovative Data Forensics to Improve Test Security
When such an implementation is used, the next question is, are both forms going to be equal in terms of content as well as difficulty? In other words, how can we ensure that both forms will be equivalent? This issue becomes exacerbated when we are not just considering Forms A and B, but Forms A, B, C, D, E, F, G, etc.
FormCast® is PSI’s proprietary Linear-On-The-Fly (LOFT) process used for automated test assembly that creates statistically equivalent forms, thus ensuring that exam difficulty, as well as content, is virtually identical throughout. The FormCast algorithm leverages user-specified exam-level statistical parameters and thresholds to accomplish this. Using FormCast helps to mitigate security concerns by generating a unique exam form for each examinee administration, while at the same time ensuring equivalence.
This can also allow for items to be in use for longer periods of time while retaining their value. Furthermore, updates to an item pool can be made more quickly and easily, as the certification program would not be reliant on static forms.
Examinees will always talk, whether in school halls or online message boards, and testing programs will always grapple with concerns of over exposure of content. Using test development and test delivery methods, such as FormCast, that help to mitigate this risk goes a long way toward helping a testing program secure their valuable examination content while maintaining a fair and positive testing experience for all examinees.
Cyrus Mirza, PhD is a psychometrician at PSI and works out of PSI’s Glendale, CA office. Cyrus assists certification clients in a number of different industries and has extensive experience with job analysis, test development and validation, data analysis, and report writing. Cyrus enjoys fostering relationships with clients and developing various selection tools that are uniquely tailored to specific jobs. Cyrus earned his PhD in Industrial Organizational Psychology from the University of Houston.
