Adverse Impact Analysis

Understanding Data, Statistics, and Risk

Scott B. Morris, Eric M. Dunleavy, Scott B. Morris, Eric M. Dunleavy

• Publisher: Routledge
• Year: 2016
• ISBN:
  9781315301419
• Edition: 1
• Topic: Psicologia
• Subtopic: Psicologia in industrie e organizzazioni

Compliance with federal equal employment opportunity regulations, including civil rights laws and affirmative action requirements, requires collection and analysis of data on disparities in employment outcomes, often referred to as adverse impact. While most human resources (HR) practitioners are familiar with basic adverse impact analysis, the courts and regulatory agencies are increasingly relying on more sophisticated methods to assess disparities. Employment data are often complicated, and can include a broad array of employment actions (e.g., selection, pay, promotion, termination), as well as data that span multiple protected groups, settings, and points in time. In the era of "big data, " the HR analyst often has access to larger and more complex data sets relevant to employment disparities. Consequently, an informed HR practitioner needs a richer understanding of the issues and methods for conducting disparity analyses.

This book brings together the diverse literature on disparity analysis, spanning work from statistics, industrial/organizational psychology, human resource management, labor economics, and law, to provide a comprehensive and integrated summary of current best practices in the field. Throughout, the description of methods is grounded in the legal context and current trends in employment litigation and the practices of federal regulatory agencies.

The book provides guidance on all phases of disparity analysis, including:

• 
  
  

• How to structure diverse and complex employment data for disparity analysis

• 
  
  

• How to conduct both basic and advanced statistical analyses on employment outcomes related to employee selection, promotion, compensation, termination, and other employment outcomes

• 
  
  

• How to interpret results in terms of both practical and statistical significance

• 
  
  

• Common practical challenges and pitfalls in disparity analysis and strategies to deal with these issues

Part 1
Introduction

1

An Introduction to Adverse Impact Measurement in the EEO Context

Eric M. Dunleavy and Scott B. Morris

Introduction

For decades, group disparities in employment outcomes, such as hiring, promotion, termination or compensation, have been a major concern in the U.S. workplace. When outcomes differ across gender, ethnicity, age or other protected groups, these differences can give rise to allegations of unfair and discriminatory employment practices, although not all disparities result from illegal employment actions. Measurement of employment disparities plays a key role in many employment discrimination lawsuits, as well as federal regulation of employment practices, and voluntary affirmative action efforts.

We use the term adverse impact analysis to broadly describe efforts to quantify disparities in employment outcomes¹ between protected groups. The term gained widespread usage after it was introduced in the Equal Employment Opportunity Commission’s (EEOC) Uniform Guidelines on Employee Selection Procedures (1978), in which adverse impact referred to disparities in selection rates resulting from the use of pre-employment tests and other screening tools. The term has long been associated with disparate impact law suits, which rely on evidence of adverse impact to establish prima facie evidence of discrimination (more on this ahead). Although the two terms are sometimes used interchangeably, we believe it is important to make a clear distinction between adverse impact, which is a statistical result quantifying the size of group disparities, and disparate impact, which is a legal framework for evaluating whether such disparities violate antidiscrimination laws. Given this broader definition, we note that adverse impact analysis has applicability to a wider range of settings than described in the Uniform Guidelines. Indeed, parts of this volume address employment outcomes other than employee selection (e.g., compensation, promotion, termination) and other theories discrimination (e.g., pattern and practice). In addition, adverse impact analyses often have utility in settings where there is no allegation of discrimination, such as proactive monitoring of employment outcomes and setting affirmative action goals.

The proper measurement of adverse impact is a nuanced topic that has prompted considerable research and debate among scholars in legal, equal employment opportunity (EEO) and social scientific disciplines (e.g., statistics, labor economics, industrial and organizational psychology). The Supreme Court first considered the topic in the 1970s, and enforcement agencies like the EEOC, the Department of Labor (DOL) and the Department of Justice (DOJ) have followed with regulatory and subregulatory guidance. Lower courts have wrestled with the intricacies of the concept for almost 40 years, and continue to disagree on a number of important issues (Gutman, Koppes & Vodvanovich, 2010). The topic has been further complicated by the changing nature of work, where the very nature of how people apply to jobs and what data are readily available to be analyzed have matured dramatically since the Supreme Court first considered the analysis of subgroup differences in employment outcomes in the 1970s (Cohen, Aamodt & Dunleavy, 2010; Dunleavy, Mueller, Buonasera Kuang & Glenn Dunleavy 2008).

The research literature on personnel selection has demonstrated an additional point worth noting. Many of the worker characteristics considered in employee selection and the tools used to make those selection decisions, all of which are designed to reflect job-relevant factors, also tend to result in differences in average scores across racial/ethnic, gender and age groups (Aamodt, 2016; Hough, Oswald & Ployhart, 2001). These statistical disparities, regardless of their cause, may lead to differences in bottom-line employment outcomes like hiring, promotion, termination and pay. As such, the research literature demonstrates that, even absent discrimination, subgroups differences are likely the norm and not the exception.

Importantly, adverse impact does not, by itself, prove illegal discrimination. However, the existence of subgroup differences may create an obligation for the employer to explain existing disparities in a pattern-or-practice scenario, or to support the use of selection tools that cause impact and consider reasonable alternatives in a disparate impact scenario. If those disparities cannot be explained or those tools cannot be shown to be job-related or consistent with business necessity in some way, their use may be considered illegal discrimination (Gutman et al., 2010). On the other hand, if adverse impact cannot be established, it will usually be difficult for groups of applicants or employees to challenge an allegedly discriminatory selection tool, policy or practice.

Given these scenarios, the stakes associated with adverse impact analyses are obviously high for both employees and employers, as well as their legal counsel and others involved in EEO policy enforcement. EEOC investigations, DOL compliance evaluations and employment discrimination litigation associated with adverse impact may incur substantial costs, both financially and by damaging an organizational reputation. Further, adverse impact statistics serve as evidence in many employment discrimination lawsuits. Consequently, it is critical for organizations to understand how to measure adverse impact and what to do when it is found.

From the human resources (HR) practitioner’s perspective, adverse impact is an important consideration for compliance and litigation, and also as a tool for affirmative action planning. Federal contractors are required to proactively conduct these types of analyses as part of their contractual agreement to do work for the federal government. As such, adverse impact and related analytics can be used to set diversity goals and evaluate the progress of affirmative action programs. Social scientists are also concerned about adverse impact from a broader societal perspective, and technical guidance from the Standards for Educational and Psychological Testing (AERA, APA & NCME, 2014) and the Principles for the Validation and Use of Employee Selection Procedures (SIOP, 2003) makes it clear that the evaluation of subgroup differences is important, and the existence of subgroup differences creates an obligation for additional research into why those differences exist.

Most professionals in the EEO community will be familiar with the basics of adverse impact analysis as described in the Uniform Guidelines; however, in practice, the measurement of adverse impact is often more complex than many realize. This notion was recently supported by a 69-member technical advisory committee (TAC) on the topic of adverse impact measurement (Cohen et al., 2010). The TAC report outlined a number of complex decisions related to structuring data for analyses, determining which individuals and employment outcomes to include in the analysis, and choosing a data analysis strategy. Further, nuances concerning the type of employment decision and the legal context may influence these choices and how to interpret results. It is exactly these complexities that this book hopes to illuminate.

Since the concept was first introduced, there has been controversy regarding the proper measurement of adverse impact (e.g., Boardman, 1979). A variety of analytic methods have been proposed in different contexts. Some strategies were codified in the Uniform Guidelines on Employee Selection Procedures (1978), while others were imported from other areas of case law, or can be found in EEO agency compliance manuals. Some methods have been thoroughly discussed in scholarly publications, while others are known only among select practitioners. All of these methods have been evaluated in the social science scholarly literature (e.g., Collins & Morris, 2008; Jacobs, Murphy & Silva, 2013; Murphy & Jacobs, 2012; Roth, Bobko & Switzer, 2006). There are meaningful differences of opinion among experts regarding what statistics should be applied in an adverse impact analysis and the data structure decisions to frame such analyses. The controversy even extends to the very utility of disparate impact theory, where both social scientists (e.g., McDaniel, Kepes & Banks, 2011) and Supreme Court justices (e.g., Justice Scalia’s concurring opinion in Ricci v. DeStefano, 2009) have opined on the matter.

There are a number of seminal resources available on the broad concept of adverse impact as an operational and research phenomena (e.g., Outtz, 2011), employment discrimination litigation (e.g., Gutman et al., 2010; Landy, 2005), the use of statistics in discrimination litigation (Baldus & Cole, 1980; Gastwirth, 1988; Paetzold & Willborn, 2015–2016), practitioner-focused primers (e.g., Biddle, 2011) and technical advisory practice-guidance (e.g., Cohen et al., 2010). However, no existing resource provides a multidisciplinary and in-depth consideration of adverse impact measurement and the complexities surrounding the topic. This book is the first edited volume devoted solely to adverse impact measurement and EEO analytics.

Our goal is to bring together the diverse literature on adverse impact measurement, spanning work from statistics, industrial/organizational psychology, human resource management, labor economics and law, to provide a comprehensive and integrated summary of contemporary practices in the field across a wide range of nuanced scenarios. As such, chapters are written by statisticians, industrial/organizational psychologists, labor econometricians, lawyers, federal compliance officers and practitioners. It is also important to note that care was taken to represent a balanced set of perspectives that (1) are involved in both proactive analyses and formal litigation support, and (2) represent both plaintiffs/government agencies and employers in a variety of situations.

Where applicable, chapters describe both basic and advanced statistical methods for modeling employment disparities, and practical issues in data preparation, analysis, interpretation and presentation. Throughout, the description of methods is grounded in the legal context and current trends in employment litigation and the practices of federal regulatory agencies. Overall, this text provides a comprehensive discussion of the various types of employment data and statistical analysis used to assess employment disparities across a wide variety of employment outcomes. We hope that this book will be a resource for professionals who are regularly involved in conducting and discussing high-stakes disparity analysis research. This audience likely includes industrial/organizational psychologists, labor economists, statisticians, lawyers and HR professionals.

Next, this chapter provides some general background information, including an overview of the legal and regulatory context, and a brief primer on statistical methods commonly used to evaluate employment outcomes.

A Primer on Legal Context: Laws, Enforcement Agencies and Discrimination²

EEO Laws

There are many EEO laws and related regulations. Table 1.1 provides a high-level overview of the most important federal EEO laws and regulations. This list is not intended to be exhaustive, and instead highlights those laws and regulations where adverse impact measurement may be an important issue. Title VII is the oldest and most consequential, and has served as the model for other regulations. This law protects against discrimination based on race, color, religion, sex and national origin. It protects individuals related to (1) terms, conditions and privileges of employment; (2) segregation and classification; and (3) retaliation.³ In other words, Title VII prohibits discrimination in any employment decision one can think of: hiring, promotion, termination, classification, assignment, training opportunity, pay and so forth.

It is important to note that Title VII protects everyone in each of the five respective categories, including males and nonminorities. As such, so-called reverse discrimination is actionable under Title VII, and adverse impact analyses may involve comparisons of any number of groups depending on context. In many situations “class action” analyses, where a group of victims is alleged, begin with a 2×2 table where employment outcomes (e.g., hired or not hired) are evaluated across two groups (e.g., men versus women, race 1 versus race 2). In many situations analyses can be expanded to evaluate whether legitimate nondiscriminatory factors explain variability in those outcomes.

Shortly after Title VII became active, Executive Order 11246 was passed in 1965. The order requires federal contractors to (1) provide equal employment opportunity to Title VII protected classes, as well as sexual orientation and gender identity and (2) provide affirmative action to previously disadvantaged groups (i.e., women and minorities). The distinction between equal employment opportunity and affirmative action under 11246 is critical, and plays an important role in structuring analyses appropriately. Equal employment opportunity is essentially nondiscrimination in...