The new 4th prong of #HRanalytics

Doing data science or analysis within HR is complicated, suddenly model accuracy is no longer the top concern. There are secondary objectives, and if overlooked they can prick you. The most discussed is that of adverse impact, which we have seen a lot on social lately around machine learning. Here are some quick examples:

Are you propagating or creating bias in your modeling techniques? Are the humans teaching the computers to behave badly? If so, how do you mitigate that and reduce that effect? 

An HR Data Science Approach [ 2 prong ] 

For data scientists who exist within HR analytics, there are two prongs they are concerned about addressing. Those are model validity and model diversity. The data scientists want the models that have the maximum predictive power (r, AUC, PR, accuracy, etc...). The second one is diversity, the model must not propagate or create an adverse impact on race, age, gender, disability, or veteran status. The objective is simple enough, the data scientist will define a number from 0 to 1 for performance validity (r, AUC, etc..) and a metric from 0 to 1 for diversity (Minority selection rate, etc...). Then they will require that someone else takes responsibility and decides what the priorities are between these two objectives. Do we only care about validity? Do we only care about diversity? For most cases, it depends. It depends on the risk tolerance of the employer and the defensibility of the performance data. 

An IO Psych Approach [ 3 prong ] 

IO Psychs have been at this game longer. They have seen and observed a vast variety of evolving class action lawsuits. There is a third prong, and that is job relatedness. You must collect features that can be defended for being job-related. A common example I like to give would be 3D accelerometer data from your mobile phone. Did you know that when you walk around with your phone in your pocket a data scientist can predict if you are walking up stairs, walking, walking down stairs, or (my personal favorite) walking and talking with someone. Even your own personal identity can be predicted from your moving phone data compared to someone else. 

The 3D Accelerometer data of your phone is rich enough to predict some surprising classifications. Suppose an HR startup was created that was able to download a candidate's 3D phone movement behavior and predict job performance from that. Sounds silly? Silly as it may be, the data set is so rich I can promise you that you would find some predictive validity. Suppose now, that you were even able to show no adverse impact, is this good enough for legal defensibility? The third prong becomes clear here where you would have a difficult time arguing that this feature set is job-related. The same arguments crop up with those chasing performance validity in social data such as Twitter, Facebook, Snapchat, or Instagram. Yes, they are predictive, but can you defend job relatedness? For these types of exotic feature sets that seem removed from the job, they can be used, but first, they have to be mapped back to a job-related competency (i.e. grit, communication, etc...). 

The problems we are all solving within HR are HARD! They are not single objective anymore, but complex multiple objective optimizations where even the weighting or priorities between these objectives become muddy and awkward. So if you build me a model with high-performance validity [check], great diversity with no adverse impact [check], job-related [check] most might assume that all bases have been covered here. In my discussions at various speaking events and polling the crowd on what is ok and what has crossed an ethics boundary I have realized there is a new 4th prong out there. I consider this 4th prong the "predetermined destiny" predictor.

Predetermined destiny, the new prong [ 4 prong ]

The moment you cross the line where you suggest a feature that is uncoachable/unchangeable and potentially predetermined the crowd no longer cares about you satisfying the 3 prongs. They are ready for torches and pitchforks. 

These types of features include things like DNA (already illegal in the US). Remember the movie Gattaca? If you haven't seen it go and watch it and hopefully you will get a sense of why this feature can never be used for screening even if you cover the other 3 prongs.

Just because DNA is off the table doesn't mean there aren't others. The most recent one I saw was electrical signals from the brain. Did you guys see the guy from the UK that was able to fly a drone with just his mind using brain signals?

Just like movement signals are being classified from the brain waves a model could be built to predict familiarity or a new metric of cognitive ability. Imagine a job screening process where there are no spoken words during the interview. You wear this brain cap and watch an animation. Your brain response tells the employer how familiar you are with the job-related content, how smart you are, and by the time the slide show is over, there is enough data to determine if you are hired? Science fiction? Not really? Messed up? Very. This approach could begin to pick up on abilities that are no longer trainable. You either have them or you don't. Want to work for Google? You'll need a different brain. Right now, we at least feel we could pivot or redirect our lives to make anything happen. We may already be dealing with some form of predetermined destiny with SAT and ACT scores for admission, an argument I heard at SIOP two years ago, but most don't see it that way. 

Curious what people think the other prongs or gotchas might be when building predictive models in HR. Besides predetermined destiny, you might argue a creep factor, if it is too creepy society might not accept it.