There are so many definitions of validity (check out one of our prior blog posts on validity), but many of my clients tend to focus on criterion-related validity.
Criterion-related validity is the notion that an assessment is meaningfully related to some outcome of interest – usually job performance. While it makes sense that clients are most interested in criterion-related studies, validity is not a unitary concept and the uniform guidelines lays out other forms of validity evidence that are also useful (Federal Register, 1978). Additionally, criterion-related studies are often not possible to conduct right away, and it is still necessary to understand the relationship of a test to the content on the job.
Therefore, this blog will dive into the content validation process and why/how its important in the assessment process.
What is content validation?
An assessment is said to be content valid when its measures are representative of all facets of the job it is meant to test for.
Why is a content validation a good idea for a production simulation?
Although production simulations are high in face validity, meaning they appear to be highly related to the job, it is important to make sure that the tasks in the production simulation are related to the tasks in the job. This is because production simulations often have cognitive and physical ability measures within them. These measures are historically associated with adverse impact, and therefore, providing a linkage between the assessment content and the job content is a necessity.
How do you conduct a content validation?
There are several steps required in a content validation:
Job Content Experts work with Subject Matter Experts (SMEs) to identify the tasks required in the job.
Job Content Experts Identify the tasks required in the production simulation.
Subject Matter Experts rate the relevancy and importance of each production simulation task.
Job Content Experts map the production simulation tasks to the job tasks and determine if there is deficiency in the mappings (meaning that all the important and relevant tasks aren’t being measured in the production simulation).
Finally, all tasks are mapped to competencies for ease of interpretation.
Content validation is an important step in building a fair, efficient, and predictive production simulation. However, as you can see, they can often be time consuming and require a lot of coordination between Job Content Experts and Subject Matter Experts. Therefore, it is important to keep this step in mind and start on this process early when implementing a production simulation.
Equal Employment Opportunity Commission, & Civil Service Commission. (1978).
Department of Labor, & Department of Justice.(1978).
Uniform Guidelines on Employee Selection Procedures. Federal Register, 43(166), 38290-38315.