<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=353110511707231&amp;ev=PageView&amp;noscript=1">

Big Data or Big Dustbowl?

December 10, 2013

The HR world is now abuzz with Big Data.  But do we have a good feel for what Big Data is and how it will be used?  According to bestselling author Bernard Marr, Big Data is “our ability to collect and analyze the vast amounts of data we are now generating in the world.”  But just because we now have the processing horsepower coupled with vast amounts of information doesn’t mean that we’re going to come up with magical solutions.  If you really think about it, in the field of selection and hiring we’ve been working with Big Data for decades, albeit maybe it wasn’t so big. describe the image

Biodata is essentially Big Data.  Biodata is any information on an individual, usually a job candidate, that relates to their personal history, background, experience, etc.  For instance, where they went to school, the degree they obtained, how many jobs they’ve had, the types of jobs, and even things such as their favorite courses.  The goal of biodata is to find a stable set of variables, drawn from information provided by applicants or incumbents, to predict meaningful outcomes such as turnover, sales quota attainment, injury risk, and a slew of other criteria.  I/O psychologists have been doing that since the turn of the century, the 20th not the 21st.  The biggest criticism of purely empirical biodata has long been that is just “dustbowl empiricism,” meaning that if it correlates it should be used, whether you know why or not.  That’s all fine and good but it is decidedly atheoretical and likely results in a lot of spurious findings that also “shrink” or disappear when they are appropriately cross-validated.  In addition, it can lead to some really crazy findings, many of which would not only be unethical or unstable but also potentially illegal.  Most I/O psychologists who use biodata now endorse more of a “rainforest empiricism” perspective, wherein there should be at least some theoretical basis for understanding correlations between two or more variables. 

My point is not that Big Data is a bad thing.  It offers many wonderful solutions in fields ranging from medicine, finance, science and transportion that we haven’t even dreamed up yet.  However, we need to be cognizant that just because you look at thousands of variables across hundreds of thousands of people doesn’t necessarily mean that you will turn up astounding findings.  What if we find out that people who eat breakfast at Dennys twice a month or more, drive SUV’s and own an iPhone tend to make better recruiters?  Would you screen out people who drive sedans, don’t eat at Dennys and us an Android?  It’s easy to fall into the trap of being mesmerized by the words Big Data.  I recently heard an interview with the president of a very large consulting firm (admittedly a competitor of ours) being interviewed about Big Data and how it is used in selection.  One of the interesting things he pointed out when asked about what Big Data had shown them was that turnover in retail environments is directly related to the length of commute the employee must make.  Turns out that people who have to commute a long way to work are more likely to quit and look for a job closer to home.  Not sure who saw that coming… except anyone who has worked in HR for retail stores in the past half century. 

It’s good to embrace the concept of Big Data and what it might hold for us.  It’s quite another thing to think that it a panacea that will solve all of our problems.  It’s not.  

Matthew O'Connell, Ph.D. Matthew O'Connell, Ph.D. was the Co-Founder and Executive Vice President of Select International, which was acquired by PSI. For more than 20 years, he was a driving force when it comes to designing, evaluating and integrating selection tools into systems that meet the specific needs of Global 2000 organizations. He is the co-author of the business best-selling book, Hiring Great People.