<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=353110511707231&amp;ev=PageView&amp;noscript=1">

Are you Hiring a Nurse or a Pirate? Face Validity in Healthcare Assessments

June 21, 2017

That’s a serious question. The answer is obvious, right? 

Maybe not...

Validity in Healthcare AssessmentsI saw a demo of a new behavioral assessment that purports to take advantage of the latest gaming technology. It was being sold as a tool for healthcare organizations. As expected, it had some basic personality and personal belief questions measuring things like dependability, empathy, and collaboration, as expected.

But then it had some complicated situational judgment questions that put the individual in a situation and ask for the best responses to that situation. In our work with nurses, for instance, we use nursing-specific situations about dealing with patients or family members.

This test, however, tried to use cutting-edge, cool-looking situations and graphics, asking the individual to assume he/she is a pirate in a video-game-like scenario and to react to the situation. It did look cool. The graphics were video game quality. I have a young nephew who might really like playing the game. But how relevant is it to nursing?

When our team builds behavioral assessments, we talk a lot about “face validity,” which is the concept that the person taking the test understands how this test is relevant to the position. When we use images or videos in our healthcare assessments, they are of healthcare-specific situations. When we ask individuals to respond to a situation, it’s the type of situation they’d face in the role. In our physician leadership tools, we put the physician in the role of a physician leading a team working on a project. For frontline staff, it might be a question posed by a patient’s family member. For a nurse, it might be a patient who is upset.

Not surprisingly, when we ask candidates to rate their experience taking our tests, the responses are quite good. The test is short, engaging and "face valid." Candidates often note that they see how it’s relevant to the role.

When evaluating selection tools, there are many variables to consider. I’d be interested to see the data showing whether the pirate game questions are actually predictive of nursing behavior on the job. I probably don’t need to see the data on whether candidates feel the tool is "face valid" – it’s not. 

Be careful when evaluating selection technology. Gamification is a hot topic and has its advantages, but measuring situational judgment in nurses probably isn’t one of them.


To learn more about important considerations in building a healthcare-specific selection system, see our free white paper, below.

New Call-to-action

Bryan Warren Bryan Warren is the President of J3 Personica, a consulting, assessment, training, and coaching firm, and a guest blogger for PSI. Bryan is an expert in progressive talent strategies, with a particular focus on leader and physician selection and development.