<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=353110511707231&amp;ev=PageView&amp;noscript=1">

Do You Have a Clearly Defined "Scoring System" for Your Hiring Process?

October 18, 2016

fantasy-football-scoring.jpgThis is Part 2 in our blog series about how fantasy football can improve your hiring process. Click here to read Part 1.

We're six weeks into the NFL season, and for fantasy football enthusiasts, you're about half-way through your regular season. With 6 games out of the way, this is the time when struggling fantasy owners are moving out of full-panic mode and into excuse-making mode. You're either bragging about your drafting prowess and taunting your fellow owners on the message boards or completely silent and trying to avoid ridicule. Yes, this is when owners either celebrate or second-guess their draft preparation strategy.

One factor owners often overlook when preparing for their draft is the scoring system of their league. Fantasy football sites offer leagues several options for customizing both the number of players you can start at each position and the way those players earn points. Some leagues only give points for yardage and touchdowns scored by your players. Other scoring systems favor the number of receptions players have with fewer points for yardage. Some leagues start 3 or more receivers. Others only start 2. Most leagues give quarterbacks 4 points for each touchdown pass thrown in a game, but I once played in a league that gave 6 points.

The specific scoring setup of your league directly impacts what kind of players will work best for your team. Players that are great in standard scoring leagues may just be average in Points Per Reception (PPR) leagues. Some players are a better fit for one scoring system than they are in others. If you go into your draft without considering the factors that will lead to success in your league, you will likely have a bad year.

The same goes for employment selection. I'm often surprised to learn how little hiring managers have thought about what skills, abilities, knowledge, and experience are required to be successful in the specific job they’re hiring for. And this can have severe consequences.

Below are a few things that hiring managers can learn from fantasy football:

Study the “Scoring System”

Managers need to know their own “scoring system” for job performance before making a selection decision. If you don't know what leads to success, you don't know what to look for when making a selection decision. Do your homework.

This is the point of the post where many of you are probably thinking to yourself, “Oh here we go… the I-O Psychologist is going to tell me I need to do a Job Analysis. Blah, blah, blah you I-Os always tell me to do a Job Analysis.”

Stay with me, here. I know most hiring managers and recruiters don’t have the time, access, or even the authority to do a job analysis. I get it.

Is a Job Analysis the best way to determine what the critical performance factors are for the job?


Do I recommend performing a job analysis to establish the “Scoring System” for your selection process?

-Of course I do.

Does that mean that if you aren’t able to do a job analysis right now, for whatever reason, there isn’t anything you can do?

-Absolutely not!

Even 15 minutes of “homework” can help. Here are my tips for identifying the success criteria for a role. Sure, some of these sound a lot like Job Analysis, but you can do these informally and on your own. If I were going to be hiring someone and didn’t have the opportunity to do a full Job Analysis, this is what I would do:

1) Start with O*NET

ONet.pngThe US Department of Labor’s database of occupational definitions is quite comprehensive, and you can likely find a detailed description of the skills, knowledge, and abilities required to perform a given job. Why reinvent the wheel? A lot of research goes into building and maintaining O*NET, and most practicing I-O psychologists (myself included) reference O*NET as a starting point during a job analysis. You should absolutely start there too.

Why I like O*NET: It is easy to fall down a rabbit hole filled with generic and uninformative descriptors when trying to define success criteria for a job. Things like “good work ethic,” and “a positive attitude,” and “integrity” come up all of the time. It’s hard to argue against those “criteria,” but they aren’t very specific and are difficult to quantify. You could say those things about any job, and one manager’s definition of “good work ethic” may be vastly different from another manager’s. That’s where O*NET is very useful. O*NET is a great way to find the language to help you specifically and clearly define the success criteria for a job.

2) Review existing materials

Most jobs have plenty of documentation you can review. A Job Description is great to have, but don’t stop there. My go-to resources are performance evaluation forms and training materials.

Why I like these sources: These two sources are invaluable when it comes to prioritizing what to look for in a candidate. If someone is trained for 3 weeks on a piece of technology, do you really want to spend a ton of time looking for their experience with similar technology? It is covered extensively in training, and therefore the experience itself isn’t all that important – move on to something else that is. Which is where Performance Evaluation forms come in. If you know how the employee’s performance will be evaluated, there’s a good chance those criteria are important. Prioritize those criteria over others that are less critical.

3) Visit the work environment

Do a site/facility tour to get a sense for the work environment as well as the equipment, technology, software, and tools used on the job.

Why I like this: Context and environment play a big role in how someone is able to perform. Is the office space open with a lot of collaboration? Are people sectioned off into offices or high-walled cubes? Is the environment fast paced? Energetic? Having that context in addition to a first-hand view of how people interact with technology, equipment, and tools is very informative.

4) Talk to Job Experts

meeting.jpgEven if you’re the manager making the selection decision, you will benefit from rounding your perspective on success criteria by asking other managers what characteristics they find critical for success. If you’re a recruiter, you should definitely talk to the hiring manager, and potentially a few of their peers to ensure you know what they’re looking for. Don’t know what to ask? Here are a few questions I’ve found effective:

  • What are the most important job responsibilities/accountabilities for the role?

  • What differentiates your star performers from your average or below-average performers? What specifically do the star performers do differently?

  • What are the common reasons people fail? What skills or work-styles did those individuals lack that caused them to fail?

  • What is the hardest part of the job for new employees?

  • What skills/knowledge does a new hire need to have on their first day in order to perform their job duties? (try to understand what is covered in training and what isn’t)

Why I like these conversations: Work evolves. What people are trying to accomplish may remain the same, but how they accomplish it can change - quickly. Having current information from Job Experts ensures you’re looking for the right thing. Quick tip – when asking questions like those listed above, probe for specific examples. Not only is this good practice for your upcoming candidate interviews, you’ll get better information. As an example, if you ask, “What is the hardest part of the job for new employees?” and the person is struggling to come up with anything, prompt them to think of the last new hire that struggled to gain job proficiency. What was challenging for them?

5) Update inaccurate or out-of-date information

Since you’re talking to Job Experts, have them review the job description and correct any inaccurate or out-of-date information (or correct them yourself). Look at certifications, skills, years of experience, software experience, etc. These tend to become outdated.

Why I do this: Job Descriptions remind me of holiday decorations. You only bring them out when you need them, and – at least for me – you don’t remember how flawed your décor is until it is too late. (e.g. “oh yeah, the cat used our Horn of Plenty centerpiece to sharpen his claws last year,” or “oops, I forgot we have a huge cranberry sauce stain on our Thanksgiving table runner.”). You tell yourself you’ll get by this year but you’ll replace them after the holiday when everything is on sale. But you don’t, and next year you do the same thing. Job descriptions aren’t thought about until it is too late, and most people know they’re not ideal, but just live with it. “Does someone really need to be proficient in Microsoft Excel 2007? I’m pretty sure we use Office 365. Hmm. I can’t remember for sure. I’ll just leave it.” Make sure it is current.


So there you have it. Take the time to define what you’re looking for in a new hire and write it down. You’ll need your list of criteria for the next important step: “Drafting” a successful team. I’ll cover that in my next post. Until then, I wish you well in your fantasy season.

For those of you that care, at this point in my fantasy season, I think I did a decent job drafting people that work well in my leagues’ various scoring systems. However, in one league, I must have overlooked the “unlikely to get hurt” criterion. Someone has to get last place, right? That’s my excuse, at least.

New Call-to-action

Lance Andrews Lance Andrews was the Director of Client Strategy for PSI. In this capacity, Lance served as a thought leader dedicated to improving the customer experience and driving revenue by articulating/translating how PSI's products and services address specific customer challenges. For more than a decade, Lance led a large number of large-scale projects involving the design, implementation and evaluation of assessment solutions across multiple jobs and within multiple industries, such as retail, financial services, manufacturing, telecommunications, healthcare, and more. His professional and research interests focus on assessment system effectiveness, legal defensibility, sustainability, and innovations in assessment methodology. Lance has a M.A. in Industrial/Organizational Psychology from Minnesota State University, Mankato, and is an active member of the Society for Industrial and Organizational Psychology (SIOP), the American Psychological Association (APA), and Minnesota Professionals for Psychology Applied to Work (MPPAW). His research on assessment and selection practices has been published in peer-reviewed journals, book chapters, and presented at annual conferences.