Big Data and HR

It was said not long ago that ‘big data’ would move HR from business support to business leadership.

In a recent article in the Harvard Business Review, Professor Peter Cappelli expressed some reservations. He argues that “there is no such thing as big data in HR”.

The hope was that HR data collected by large employers and subjected to big data analytical processes would reveal valuable insights into such critical questions as what type of person to hire for a given role or who make the best business leaders.

The challenges of big data analytics in HR:

  • Professor Cappelli points out that most global companies have thousands of employees and not millions. Datasets of this size, he says, do not qualify as ‘big data’ at all.
  • Most companies still collect ‘observations’ about employees on an annual basis, which further significantly reduces the dataset available for analysis.
  • Many companies have different tasks (and their related data) stored in different databases which are not compatible. Making them compatible for analysis requires time, investment and commitment. Companies would need to be confident that the return on their investment in achieving compatibility would be valuable to their business. The evidence for this value is still far from compelling.
  • Questions such as who to hire or promote and how to assess or improve individual performance have been closely studied for most of the 20th century. Professor Cappelli thinks that the chance that big data analytics will come up with some profound new HR insight is very small indeed.

Then, there are the privacy issues:

  • Privacy laws pretty much everywhere restrict the use of personal and sensitive personal data. In addition, some countries (including all in the EU) restrict the international transfer of data outside their jurisdictions without safeguards. Fines for breaches in the EU will rise sharply in 2018.
  • Restrictions on both use and transfer of data can prevent global companies from pooling multi-jurisdictional datasets, again limiting the size of the available data and therefore the value of the data submitted to analysis.
  • Some countries require ‘freely given’ employee consent to the use and transfer of their personal data. Given the perceived imbalance of power between employer and employee, Court judgments in some EU countries make ‘freely given’ employee consent very hard to obtain.
  • Encryption may help maintain data privacy compliance if it secures real anonymity, but it comes with a financial and operational cost.

And then, there is the discrimination issue:

  • If a large dataset analysis shows that people of a particular gender, or ethnic group or age range make the best performers for a given role then either the information is ignored (and is therefore worthless) or the company changes its recruitment/promotion policies and submits itself to the risk of discrimination claims no doubt supported by the very existence of the HR dataset analysis.

And so, if the Professor is right, it seems that algorithms, for most companies, are not for now at least going to be much help answering the everyday questions of whom to hire or whom to promote.

HR will have to keep figuring it out for their business partners in the traditional way.