Handbook of Regression Modeling in People Analytics

With Examples in R and Python

Keith McNulty

• Publisher: Chapman and Hall/CRC
• Year: 2021
• Print ISBN:
  9781032046631
  9781032041742
• Edition: 1
• eBook ISBN:
  9781000427929
• Topic: Mathematics
• Subtopic: Human Resource Management
• Index: Mathematics

Despite the recent rapid growth in machine learning and predictive analytics, many of the statistical questions that are faced by researchers and practitioners still involve explaining why something is happening. Regression analysis is the best 'swiss army knife' we have for answering these kinds of questions.

This book is a learning resource on inferential statistics and regression analysis. It teaches how to do a wide range of statistical analyses in both R and in Python, ranging from simple hypothesis testing to advanced multivariate modelling. Although it is primarily focused on examples related to the analysis of people and talent, the methods easily transfer to any discipline. The book hits a 'sweet spot' where there is just enough mathematical theory to support a strong understanding of the methods, but with a step-by-step guide and easily reproducible examples and code, so that the methods can be put into practice immediately. This makes the book accessible to a wide readership, from public and private sector analysts and practitioners to students and researchers.

Key Features:

• 16 accompanying datasets across a wide range of contexts (e.g. academic, corporate, sports, marketing)
• Clear step-by-step instructions on executing the analyses
• Clear guidance on how to interpret results
• Primary instruction in R but added sections for Python coders
• Discussion exercises and data exercises for each of the main chapters
• Final chapter of practice material and datasets ideal for class homework or project work.

1

The Importance of Regression in People Analytics

DOI: 10.1201/9781003194156-1

In the 19th century, when Francis Galton first used the term ‘regression’ to describe a statistical phenomenon (see Chapter 4), little did he know how important that term would be today. Many of the most powerful tools of statistical inference that we now have at our disposal can be traced back to the types of early analysis that Galton and his contemporaries were engaged in. The sheer number of different regression-related methodologies and variants that are available to researchers and practitioners today is mind-boggling, and there are still rich veins of ongoing research that are focused on defining and refining new forms of regression to tackle new problems.

Neither could Galton have imagined the advent of the age of data we now live in. Those of us (like me) who entered the world of work even as recently as 20 years ago remember a time when most problems could not be expected to be solved using a data-driven approach, because there simply was no data. Things are very different now, with data being collected and processed all around us and available to use as direct or indirect measures of the phenomena we are interested in.

Along with the growth in data that we have seen in recent years, we have also seen a rapid growth in the availability of statistical tools—open source and free to use—that fundamentally change how we go about analytics. Gone are the clunky, complex, repeated steps on calculators or spreadsheets. In their place are lean statistical programming languages that can implement a regression analysis in milliseconds with a single line of code, allowing us to easily run and reproduce multivariate analysis at scale.

So given that we have access to well-developed methodology, rich sources of data and readily accessible tools, it is somewhat surprising that many analytics practitioners have a limited knowledge and understanding of regression and its applications. The aim of this book is to encourage inexperienced analytics practitioners to ‘dip their toes’ further into the wide and varied world of regression in order to deliver more targeted and precise insights to their organizations and stakeholders on the problems they are most interested in. While the primary subject matter focus of this book is the analysis of people-related phenomena, the material is easily and naturally transferable to other disciplines. Therefore this book can be regarded as a practical introduction to a wide range of regression methods for any analytics student or practitioner.

It is my firm belief that all people analytics professionals should have a strong understanding of regression models and how to implement and interpret them in practice, and my aim with this book is to provide those who need it with help in getting there. In this chapter we will set the scene for the technical learning in the remainder of the book by outlining the relevance of regression models in people analytics practice. We also touch on some general inferential modeling theory to set a context for later chapters, and we provide a preview of the contents, structure and learning objectives of this book.

1.1 Why is regression modeling so important in people analytics?

People analytics involves the study of the behaviors and characteristics of people or groups in relation to important business, organizational or institutional outcomes. This can involve both qualitative methods and quantitative methods, but if data is available related to a particular topic of interest, then quantitative methods are almost always considered important. With such a specific focus on outcomes, any analyst working in people analytics will frequently need to model these outcomes both to understand what influences them and to potentially predict them in the future.

Modeling an outcome with the primary goal of understanding what influences it can be quite a different matter to modeling an outcome with the primary goal of predicting if it will happen in the future. If we need to understand what influences an outcome, we need to get inside a model and construct a formula or structure to infer how each variable acts on that outcome, we need to get a sense of which variables are meaningful or not, and we need to quantify the ‘explainability’ of the outcome based on our variables. If our primary aim is to predict the outcome, getting inside the model is less important because we don't have to explain the outcome, we just need to be confident that it predicts accurately.

A model constructed to understand an outcome is often called an inferential model. Regression models are the most well-known and well-used inferential models available, providing a wide range of measures and insights that help us explain the relationship between our input variables and our outcome of interest, as we shall see in later chapters of this book.

The current reality in the field of people analytics is that inferential models are more required than predictive models. There are two reasons for this. First, data sets in people analytics are rarely large enough to facilitate satisfactory prediction accuracy, and so attention is usually shifted to inference for this reason alone. Second, in the field of people analytics, decisions often have a real impact on individuals. Therefore, even in the rare situations where accurate predictive modeling is attainable, stakeholders are unlikely to trust the output and bear the consequences of predictive models without some sort of elementary understanding of how the predictions are generated. This requires the analyst to consider inference power as well as predictive accuracy in selecting their modeling approach. Again, many regression models come to the fore because they are commonly able to provide both inferential and predictive value.

Finally, the growing importance of evidence-based practice in many clinical and professional fields has generated a need for more advanced modeling skills to satisfy rising demand for quantitative evidence from decision makers. In people-related fields such as human resources, many varieties of specialized regression-based models such as survival models or latent variable models have crossed from academic and clinical settings into business settings in recent years, and there is an increasing need for qualified individuals who understand and can implement and interpret these models in practice.

1.2 What do we mean by ‘modeling’?

The term ‘modeling’ has a very wide range of meaning in everyday life and work. In this book we are focused on inferential modeling, and we define that as a specific form of statistical learning, which tries to discover and understand a mathematical relationship between a set of measurements of certain constructs and a measurement of an outcome of interest, based on a sample of data on each. Modeling is both a concept and a process.

1.2.1 The theory of inferential modeling

We will start with a theoretical description and then provide a real example from a later chapter to illustrate.

Imagine we have a population $\mathcal{P}$ for which we believe there may be a non-random relationship between a certain construct or set of constructs $\mathcal{C}$ and a certain measurable outcome $\mathcal{O}$. Imagine that for a certain sample S of observations from $\mathcal{P}$, we have a collection of data which we believe measure $\mathcal{C}$ to some acceptable level of accuracy, and for which we also have a measure of the outcome $\mathcal{O}$.

By convention, we denote the set of data that measure $\mathcal{C}$ on our sample S as $X=x_1,x_2,\dots ,x_p$, where each x_i is a vector (or column) of data measuring at least one of the constructs in $\mathcal{C}$. We denote the set of data that measure $\mathcal{O}$ on our sample set S as y. An upper-case X is used because the expectation is that there will be several columns of data measuring our constructs, and a lower-case y is used because the expectation is that the outcome is a single column.

Inferential modeling is the process of learning about a relationship (or lack of relationship) between the data in X and y and using that to describe a relationship (or lack of relationship) between our constructs $\mathcal{C}$ and our outcome $\mathcal{O}$ that is valid to a high degree of statistical certainty on the population $\mathcal{P}$.

This process may include:

• Testing a proposed mathematical relationship in the form of a function, structure or iterative method
• Comparing that relationship against other proposed relationships
• Describing the relationship statistically
• Determining whether the relationship (or certain elements of it) can be generalized from the sample set S to the population $\mathcal{P}$

When we test a relationship between X and y, we acknowledge that data and measurements are imperfect and so each observation in our sample S may contain random error that we cannot control. Therefore we define our relationship as:

where f is some transformation or function of the data in X and ϵ is a random, uncontrollable error.

f can take the form of a predetermined functi...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Contents
• Foreword by Alexis Fink
• Introduction
• 1 The Importance of Regression in People Analytics
• 2 The Basics of the R Programming Language
• 3 Statistics Foundations
• 4 Linear Regression for Continuous Outcomes
• 5 Binomial Logistic Regression for Binary Outcomes
• 6 Multinomial Logistic Regression for Nominal Category Outcomes
• 7 Proportional Odds Logistic Regression for Ordered Category Outcomes
• 8 Modeling Explicit and Latent Hierarchy in Data
• 9 Survival Analysis for Modeling Singular Events Over Time
• 10 Alternative Technical Approaches in R and Python
• 11 Power Analysis to Estimate Required Sample Sizes for Modeling
• 12 Further Exercises for Practice
• References
• Glossary
• Index