Several years ago I was doing some work for a manufacturing company. This company was completely overhauling its hiring process for their entry-level assembler positions. During the initial phase of this project, I met with a lot of front-line supervisors to learn about the assembler positions. I also used these meetings as an opportunity to discuss the implementation of a new testing and hiring procedure. One supervisor provided me some memorable advice. I’ve paraphrased his suggestion below:
We should ask applicants to make a bowl of cereal. I want to know if a person can figure out how to take down the box. Open the box. Take out a bowl. Pour the cereal in the bowl. Pour the milk. Use the spoon. I’m not even kidding. We have people that work here that I doubt could make a bowl of cereal. No common sense. They can’t follow the simplest directions. We do everything for them. They don’t know and don’t care.
We did create a testing process that measured the competencies necessary to be an effective assembler at this company. (No, the hiring process did not include a bowl of cereal.) This supervisor’s comments reflect the desire and reasoning that motivates supervisors to create their own pre-employment tests. Supervisors want reassurance that the people hired for their team have the basic skills and aptitude to meet the job requirements. They don’t want a person on their team that can’t figure out how to make a bowl of cereal. Many supervisors work for companies that don’t have a well-designed or structured selection process, so they take matters into their own hands. I have seen my fair share of homegrown, supervisor created tests over the years. Some of them are well thought out, job relevant, and seemingly effective. I appreciate the initiative of these supervisors and their desire to add a little “quality assurance” to their company’s hiring process. However if not done correctly, which is often the case, supervisor created tests can be more of a problem than a solution. In fact, more than likely you will have an inaccurate, unreliable testing process with increased legal risk. Below are 5 things to ask about your homegrown, supervisor created test:
Is it job relevant? The bowl of cereal test would probably not meet this criterion. If I was a production supervisor, I probably wouldn’t want a person assembling high performance components that couldn’t assemble a bowl of cereal. I understand the sentiment. However, a test must have well defined competencies or skills that it measures, and those competencies and skills must be demonstrably linked to the job.
Is the supervisor test a well structured, consistent process? How does the supervisor know when a person passes or fails? Is the scoring method a well defined and objective process? A common problem with supervisor tests is that the applicants are evaluated on poorly defined and subjective criteria. A supervisor ends up comparing applicants to each other as opposed to a well defined standard. This calls into question the consistency or fairness of the test. It is always critical that the testing process and criteria be consistently applied to all eligible applicants. A test that is inconsistently used is a very real legal risk for the company.
Can a minimally qualified applicant pass the test? The answer better be “yes”. We don’t want to hold applicants to a higher skill level than that level required to perform the basic job duties. Additionally, we don’t want to test people on skills that will be trained on the job. The test should measure those basic competencies or skills that new hires should already possess when they start their first day on the job.
Has the supervisor test been validated? A beneficial feature of most supervisor created tests is that they possess face validity. In other words, at face value the test appears to measure the skills needed for successful job performance. Most supervisor tests are essentially a work sample. The supervisor took a basic task that is done on the job and turned it into a test. An applicant for a Copy Proofreader job asked to identify all the spelling and grammatical errors in a document would be a good example of a face valid test. In addition to face validity, it is important that the test has been content validated by multiple job content experts or validated with a concurrent study involving current employees in the target job. One supervisor’s opinion is not a persuasive argument for the validity of a test.
Where is the documentation? Any test that faces legal scrutiny will need to have its design, development, measurement methods, criteria and validation well documented.
An expert in pre-employment test development will ensure that you create an accurate and legally defensible hiring process. Don’t let your supervisors create a test without the necessary guidance of an I/O Psychologist or other test development specialist. Certainly, supervisors, as job content experts, play an important role in creating an applied or “hands on” test. However, they will need an expert to make sure that the above questions are answered and addressed appropriately to ensure that the test is accurate, reliable, and fair.