One of the first things we do when engaging with new clients is to discuss the hiring process and determine what steps to include in their hiring process. In order to get a complete picture of a candidate, we often include several different stages and assessments. For example, we might include an initial application, an in-depth online assessment, a work sample exercise, and an interview as part of the process. Pulling together all of this information, we can see how the candidate performs on important competencies in different types of assessment methods.
When determining which steps are most appropriate for the position and the organization, clients often ask, is more always better? Does having more stages in the process provide a greater return on investment? The answer is … it depends. In answering this question, we often look at incremental validity. Before talking specifically about incremental validity, let me provide a refresher on what validity is.
How validity helps answer this question
Overall, when examining the impact and effectiveness of a specific tool or assessment, we determine its validity. Validity, or validation evidence, gives us an indication of how well the assessment measures what it is intended to measure. For example, if a tool is intended to measure cognitive ability, does it give us an accurate representation of how smart someone is in the real world? In the domain of hiring and selection, we seek tools that are accurate because that will enable us to be more confident in selecting individuals who are more likely to be successful on the job. Therefore, we want a test that has a lot of validation evidence.
Taking this a step further, when we include multiple tools in the hiring process, we want to do more than just look at the validity of each tool. Validity evidence can tell us how effective the tool is independent of other tools. However, when tools are used in combination with one another, we need to look at the validity evidence from a different perspective. All of the tools might have high validity estimates independent from each other, but when used in combination, we need to ask whether there is value added. Determining incremental validity is the way to do that.
What is incremental validity?
Incremental validity will tell us how much unique information the additional tool will have. If there is a lot of overlap in the content and sections in two assessments, then the incremental validity will be fairly low for the second tool. In these two tools, you are capturing much of the same information and the tests wouldn’t provide unique information. This would not be efficient for your hiring process from a time or resources perspective.
Alternatively, if there is little overlap in the content, then there is the potential for much greater incremental validity. For example, if we are focused on leadership skills in one tool and technical ability in another, then there is very little overlap in the content. If both tools have strong validity evidence to begin, then adding the second tool would provide a lot of unique information to help you make a more informed decision. Combining these tools in the assessment process would provide a good return on investment.
It’s more typical that you will find some degree of overlap between tools in a hiring process. A best practice of any hiring process is to measure skills in different ways to get a clearer and more accurate picture. For example, we might measure coaching skills through situational judgment items or through a role play exercise. Even though the exercises are assessing the same skill, the skills are measured in different ways. As such, it is very likely that each of these tools would provide valuable information.
When developing a hiring process, incremental validity is something important to consider. We want to make sure that each of the tools included in the process provides unique information. Unique information means a greater return on your investment. By focusing on effectiveness and efficiency of the hiring solution, we can increase the likelihood that you will find and retain top talent.