<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=353110511707231&amp;ev=PageView&amp;noscript=1">

Should You Use Oddball Interview Questions in Your Hiring Process?

April 14, 2015

interviewingConsider the following questions:

  • If you were asked to unload a 747 full of jellybeans, what would you do?

  • If you woke up and had 2,000 unread emails and could only answer 300 of them, how would you choose which ones to answer?

  • How many gas stations are there in America?

Do these sound like the typical questions included in an interview process? Definitely not, but these non-traditional interview questions are becoming more popular, especially in innovative organizations. These types of questions can include brainteasers, puzzles, or any off-the-wall types of questions. The idea behind these questions is to assess a candidate’s thought process while answering the questions. Microsoft has been credited for spearheading the use of these types of questions in the 1990s to measure a candidate’s mental flexibility, entrepreneurial potential, and creativity. Since then, several more companies have started infusing them into their hiring process. But, what do we know about these questions and how does it fit within the realm of our recommendation to use structured, behavior-based interviews?

Well, the short answer to that question is that we don’t know much. Very little research has been established on their effectiveness and utility. In an upcoming presentation at the annual conference for Society for Industrial Organizational Psychology (SIOP), Nathan Wiita, and his colleagues Elnora Kelly, R. Patrick Bradshaw, and Rustin Meyer will be sharing some results of a study looking at the use of non-traditional interview questions. In the study, working adults were asked to rate their perceptions of organizations using different types of interview conditions. Results suggest that participants perceived organizations using non-traditional interview questions as being more innovative than those organizations using structured interviews. Furthermore, they rated their anticipated job satisfaction to be higher at organizations using these non-traditional questions.

On the other hand, participants anticipated lower job success, less attraction, and less of a fit with organizations using non-traditional interview questions than organizations using structured interviews. Importantly, the researchers also found that these types of questions were much more unfair than structured interview questions. These negatives beg the question of how well these questions actually do in predicting performance.

In another upcoming presentation at SIOP related to this topic, Renee Payne, Tina Malm, Melissa Harrell from Google, Inc., will be sharing some results of an internal study conducted on how well brainteaser interview questions predict performance. They found that high performing employees did not perform any better or worse than others on the brainteaser questions. This suggests that they were not helpful in differentiating between poor and excellent performers at Google. While this is only one study, it reinforces the notion that it’s critical to make sure that before you implement any new process or step in a hiring process that you make sure that they add value by effectively predicting performance on the job.

Whether you are using these types of questions or not, it’s important to consider the following factors when developing and implementing your interview process:

  • Conduct a job analysis to determine what skills and abilities are critical for the job position. Only skills that are necessary and important for successful performance should be considered.

  • Develop targeted, behavior-based questions that measure skills deemed to be important in the job analysis.

  • Develop and use systematic scoring criteria that provide behavioral descriptors of what would be unacceptable, acceptable, and very acceptable actions.

  • Review these questions and scoring criteria with job content experts to ensure that they are relevant to the job.

  • Train hiring managers and other interviewers on the best practices of conducting a structured interview. Familiarize them with the process, skills being assessed, how to collect complete responses, and how to make ratings using the scoring criteria.

  • Be consistent with the interview process so all job candidates applying for the same position are being measured on the same skills and being assessed using the same criteria.

Interviewer Tips

Alissa Parr, Ph.D. Alissa Parr, Ph.D. is a Senior Consultant at PSI. Her areas of expertise include the development, implementation, and evaluation of assessment processes. Alissa has experience managing entry-level through executive level assessment and selection efforts across a number of different industries including government, financial, military, education, healthcare, and manufacturing.