Modern Research Methods for the Study of Behavior in Organizations

Jose M. Cortina, Ronald S. Landis, Jose M. Cortina, Ronald S. Landis

• Publisher: Routledge
• Year: 2013
• ISBN:
  9781135068455
• Edition: 1
• Topic: Bildung
• Subtopic: Experimentelle Methoden in der Bildung

Thegoalof the chapters in this SIOP Organizational Frontiers Series volume is to challenge researchers to break away from the rote application of traditional methodologies and to capitalize upon the wealth of data collection and analytic strategies available to them.In that spirit, many of the chapters in this book deal with methodologies that encourage organizational scientists to re-conceptualize phenomena of interest (e.g., experience sampling, catastrophe modeling), employ novel data collection strategies (e.g., data mining, Petri nets), and/or apply sophisticated analytic techniques (e.g., latent class analysis).The editors believe that these chapters provide compelling solutions for the complex problems faced by organizational researchers.

1

Introduction: Transforming Our Field by Transforming its Methods

Jose M. Cortina and Ronald S. Landis

Those who study human behavior in organizations confront a plethora of challenges. In order to meet these challenges, researchers sometimes employ complex measurement or analytic techniques, without necessarily knowing how, or even if, they serve the researcher’s purposes. Although there are many ways in which human–computer interaction has changed for the better, the ability to collect or analyze data without knowing what one is doing is not one of them. What we need is a sort of methodological prism that breaks our techniques into their component parts, allowing us to understand how they fit together.

Our goal for the chapters in this book is to challenge researchers to break away from the rote application of traditional methodologies and to capitalize upon the wealth of data-collection and analytic strategies available to them. In that spirit, many of the chapters in this book deal with methodologies that encourage organizational scientists to reconceptualize phenomena of interest (e.g., experience sampling, catastrophe modeling), employ novel data-collection strategies (e.g., data mining, Petri nets), and/or apply sophisticated analytic techniques (e.g., latent class analysis). We believe that these chapters provide compelling solutions for the complex problems faced by organizational researchers, problems that, if left unaddressed, might leave us on the dark side of the moon.

Too Many Cooks, too Few Appliances

The methods that we use to collect data necessarily influence (and constrain) the way that we conceptualize organizational phenomena. As a result, scientific advancements are limited, to the extent that we continue to rely on the same old methods to study new problems. Imagine a chef who wishes to make a tasty meal. If the chef is given only, say, a deep fryer with which to work, culinary options become necessarily limited. Although it is certainly true that the deep fryer will be useful for making some dishes, the chef will be in trouble if he or she would like to poach an egg [Editor’s note: Do not drop an egg into a deep fryer unless you enjoy third-degree burns]. As anyone who grew up in the deep south can tell you, the chef operating in the deep fryer-only kitchen will come to view available dishes primarily through the lens of this tool [Editor’s note: If you find yourself in Louisiana, avoid the fried green salad]. On the other hand, if the chef is provided with a range, oven, grill, wok, etc., a much wider variety of dishes can be conceptualized and executed. The same is true for the organizational researcher who operates in, say, the “OLS regression kitchen.” If ordinary least squares (OLS) serves as the only methodological tool, the researcher will come to view organizational problems through the OLS lens. Although many wonderful dishes can be made with OLS regression, many others cannot. One must limit oneself to the prediction of continuous dependent variables whose errors are uncorrelated, using variables that are, or can be, converted into interval level variables. One must restrict oneself to the study of phenomena that change in a continuous fashion over time. At a broader level, one must restrict oneself to phenomena that are sufficiently understood that one knows which questions to ask (i.e., quantitative as opposed to qualitative research). It is only when the list of tools is augmented that the list of topics can be expanded.

Of course, we have no desire to denigrate OLS regression. Indeed, there are still many social scientists who work in the even more rustic analytic kitchen in which analysis of variance (ANOVA) is the technique of choice. When confronted with the horror that is a continuous predictor, these poor devils either artificially categorize it, resulting in nonlinear, nonrandom measurement error, or relegate it to (additive) covariate status in ANCOVA. They need their blender to frappé, but, alas, it only has on/off. And don’t get us started on what is happening in the t-test galley.

The chapters compiled in this book help organizational researchers to become aware of, and appreciate, the tools that are hiding in the methods pantry. The authors of these chapters not only provide descriptions of these contemporary methodologies, but also provide examples of how they may be applied to organizational phenomena. In particular, we believe this latter aspect of each chapter may be this volume’s greatest asset. Frequently, we see researchers get excited by particular techniques, only to become frustrated because they do not see how the methods can be applied to their own work. The authors of the chapters in this volume have taken care to provide this information.

A second theme that we have attempted to integrate in the current collection of chapters is that of organizational research as increasingly complex and challenging. As a field, we study phenomena that are typically directly unobservable, temporally volatile, and in contexts that often do not permit tight, experimental control. Thus, despite the claim that ours is a field of “soft” scientists, we hope that the current chapters convince you that our field can apply rigorous methodologies for studying organizational behavior, and that, through the use of these methods, our field can further develop as an applied science that meaningfully contributes to the understanding of modern organizational phenomena and problems.

We also want to emphasize that statistics and methods are as vibrant and vital a research area as any substantive one. Both of us have had interactions with colleagues, the nature of which will be familiar to many readers of this book:

Colleague:	So, tell me, what is your primary research area of interest?
Jose/Ron:	Research methodology.
Colleague:	That isn’t an area of research.

To us, this type of interchange reflects a conceptualization of methods as immutable (read: stagnant) and leads to a cookbook application of old techniques that constrains theoretical development and knowledge creation. We believe the chapters in this volume challenge that view of methodology and, instead, convey the important contributions made by those working in the area.

Advanced, not Magical

In choosing authors and topics for this particular volume, we had certain principles in mind. First, we wanted chapters on cutting-edge topics and authors with the expertise to write them. Second, we wanted the chapters to inform and educate readers about the nature and relevance of particular techniques and tools through clear summaries and reviews. Third, we wanted the chapters to provide sufficient information to allow the reader to adapt the techniques to his or her own research. All too frequently, beneficial methodological techniques are not adopted, because researchers don’t have a clear road map for application. Finally, we wanted the chapters to prompt researchers, not only to apply newer techniques (when appropriate), but also to challenge status quo thinking about particular organizational phenomena. As a result, we specifically asked the contributors to identify cutting-edge issues with respect to particular methods that will serve to stimulate future substantive research. Our contributors have provided such a resource, and we trust that the following chapters will serve as catalysts for significant advances in the organizational sciences.

Connecting the Present (and Future) to the Past

More than a decade has passed since the publication of the most recent volume in the Organizational Frontiers Series, devoted to research methods. Since the publication of that volume (Drasgow & Schmitt, 2002), our field has seen an explosion of interest in, and use of, advanced measurement, design, and analysis techniques. At the time that Drasgow and Schmitt went to press, many of the techniques that now seem common were either in their infancy (e.g., latent growth modeling (LGM), grounded theory, response surface methodology) or so uncommon in the organizational sciences as to be unworthy of inclusion in a volume on methodology (e.g., catastrophe modeling, latent class analysis, experience sampling). Indeed, the Drasgow and Schmitt volume was instrumental in solidifying researchers’ understanding of many advanced methodological techniques, which in turn led to these techniques being more commonly and appropriately used.

We believe that our field is now poised to take another important step down the path of sophisticated methodological techniques. In recent decades, techniques have emerged that, not only improve our ability to collect data and to evaluate the data that we collect, but also provide researchers with the freedom to develop more nuanced theory. Instead of exploring LGM at a broader level, as did David Chan in his excellent and crucial chapter from the Drasgow and Schmitt volume, we assert that our field is ready to explore extensions of LGM (Ployhart and Kim), as well as pitfalls that are well understood in other fields but new to ours (Braun, Kuljanin, and DeShon). We must go beyond a descriptive treatment of grounded theory and explore the latest in case studies, textual analysis, and other quantitative methods (Gephart). We must acknowledge the existence of nonmonotonic relationships and more explicitly consider discontinuous relationships with techniques such as catastrophe modeling (Guastello) and discontinuous growth modeling (Ployhart and Kim). We should move beyond recognizing that some organizational phenomena involve hierarchical structures and parallel processes and more appropriately model these contexts (Coovert), as well as more explicitly consider individuals as part of larger systems (Kalish). In short, it is time for our field to explore the next frontiers of research methodology. Some of these frontiers may represent fine-tuning of our techniques, but others (e.g., catastrophe modeling, experience sampling) have the potential to turn our field on its ear and, indeed, have already done so (e.g., Guastello, 1988; Ilies, Scott, & Judge, 2006; Guastello, 2007).

Organization of the Volume

This book is divided into two parts: Statistical Analysis and Research Design and Measurement. In the first chapter of the Statistical Analysis part, Guastello describes catastrophe theory and the analyses that accompany it. Many of us have heard of catastrophe theory (or at least have heard of related concepts such as chaos theory), but few of us have taken steps toward applying it to our research in organizations. This is a terrible shame, because so many organizational phenomena are likely to conform to catastrophe models. In fact, we believe that our field is on the cusp (if you will) of a “catastrophe revolution,” and those who join it early will be remembered for (and credited with) having changed our field for the better.

Catastrophe models describe discontinuous phenomena, that is, phenomena that involve sudden “catastrophic” change. For example, Guastello (1987) suggested that, for low levels of task variety, there is a mono tonic, positive relationship between ability and performance, whereas, for high levels of task variety, there is a discontinuous relationship between ability and performance such that performance is stable and low for lower ability levels but tends to jump “catastrophically” at some middle level of ability, with the jump point being tied to the reward system. The jump is consistent with the tenets of insight learning, in which an “a-ha moment” creates a qualitative change in knowledge. As another example, Guastello (1988) showed that, for large workgroups, accidents are monotonically and positively related to environmental hazards. For small workgroups, however, accidents are discontinuously related to hazards, such that accident rates are stable and low for low-hazard groups, stable and high for high-hazard groups, and jump catastrophically at some mid level of hazard. One reason for this is that small groups tend to be more cohesive, and this cohesiveness creates a cyclical process that causes accident rates to “shift gears” at some level of environmental hazard.

Guastello and his colleagues have used catastrophe theory to explain a wide variety of organizational phenomena, and yet few other researchers have done so. We suspect that the reason is that most organizational researchers are intimidated by the abstruse mechanics of catastrophe modeling. In Guastello’s chapter in the current volume, he provides a detailed and approachable description of catastrophe modeling and its application. We cannot imagine a better presentation of this material and believe the chapter will serve as foundation for “catastrophically” influencing our field for the better.

In the second statistical-analysis chapter, Ployhart and Kim tackle random coefficient models (RCMs). These authors focus their attention on a surprisingly underutilized application of RCM, namely dynamic or time-varying predictor models. Although RCM and LGM have become quite common in organizational research, it is relatively rare to see research in which Level 1 predictors vary over time. And yet, as Ployhart and Kim put it, “wouldn’t it be exciting to see research showing how changes in knowledge acquisition relate to changes in job performance over time?” We know from cross-sectional research that those with greater amounts of knowledge tend to have better performance evaluations, but, because dynamic predictor models have not been applied to the problem, we don’t actually know if one’s performance increases as one’s knowledge increases! Ployhart and Kim explain the mechanics of dynamic predictor models (including latent growth models), their data requirements, the pitfalls associated with such models, and the strategies that can be used to avoid these pitfalls.

These authors also discuss extensions of the standard dynamic predictor RCM. First, they discuss lagged growth models, in which data points are lagged in time to reflect hypothesized causal sequences. Collecting data in this way, as the authors explain, allows one to address problems that are common to dynamic models, such as reciprocal causation and spurious relationships. Second, these authors describe autoregressive latent trajectory (ALT) models. In ALT models, change over time in a given variable is estimated after controlling for previous levels of the variable (i.e., the autoregressive element). As the authors point out, ALT models reflect the axiom that the best predictor of future behavior is past behavior.

Third, Ployhart and Kim discuss nonlinear and discontinuous growth models. Nonlinear growth models capture change over time as a nonlinear function of time. For example, we know that knowledge acquisition does not change in a linear manner over time, so why should its effects be modeled as if it did? Discontinuous growth models can be used to model phenomena that do not change in a monotonic manner. Indeed, discontinuous growth models are very similar to the catastrophe models described in the Guastello chapter.

Finally, Ployhart and Kim describe between groups change models. In such models, grouping variables are used to distinguish different clusters of change patterns. For well-defined groups, multiple group LGM is quite useful. For less defined or unknown groups, latent class analysis or, more broadly, mixture modeling can be used. In short, if you want to understand the latest in RCM with time-varying predictors, this chapter is a must.

Social network analysis, as described in the third statistical chapter, by Kalish, holds great promise for researchers interested in modeling social influence and communication in organizational contexts. No matter their formal structures, services provided, or products generated, organizations are fundamentally social systems. Individual employees interact with customers, colleagues, subordinates, supervisors, and myriad others through the formal and informal aspects of contemporary jobs. Unfor tunately, we organizational scientists frequently choose to simplify these complex relationships and, all too frequently, study individuals in isolation, or at best as members of collectives, and attempt to explain behavior through a somewhat static lens. Social network analysis provides us with opportunities to uncover how individual relationships (dyads, triads, etc.) are formed, influence individual behavior, and ultimately change and dissolve.

One would not likely choose to study a family by individually surveying the children and presuming that these individual perceptions fully capture the complexity of the family dynamic. Even if we were to take a higher-level perspective and consider the children as a “team,” we are still likely to miss important dyadic relationships between the children and/or the parents. Similarly, organizational researchers should not ignore the contextual aspects of modern organizations. These context...