Introduction to Modern Modelling Methods

D. Betsy McCoach, Dakota Cintron, Author

• Publisher: SAGE Publications Ltd
• Year: 2022
• Print ISBN: 9781526424037
• eBook ISBN: 9781529711080
• Edition: 1
• Topic: Social Sciences
• Subtopic: Social Science Research & Methodology
• Index: Social Sciences

Using simple and direct language, this concise text provides practical guidance on a wide range of modeling methods and techniques for use with quantitative data. It covers:

·       2-level Multilevel Models

·       Structural Equation Modeling (SEM)

·       Longitudinal Modeling using multilevel and SEM techniques

·       Combining organizational and longitudinal models

Part of The SAGE Quantitative Research Kit, this book will give you the know-how and confidence needed to succeed on your quantitative research journey.

1 Clustering and Dependence: Our Entry Into Multilevel Modelling

Chapter Overview

• Nested data and non-independence 3
• Intra-class correlation coefficient 6
• Effective sample size 8
• Computing the effective sample size n^eff 9
• Further Reading 13

Frequently in the social sciences, our data are nested, clustered or hierarchical in nature: individual observations are nested within a hierarchical structure. ‘The existence of such data hierarchies is neither accidental nor ignorable’ (Goldstein, 2011, p. 1). Examples of naturally occurring hierarchies include students nested within classrooms, teachers nested within schools, schools nested within districts, children nested within families, patients nested within hospitals, workers nested within companies, husbands and wives nested within couples (dyads) or even observations across time nested within individuals. ‘Once you know that hierarchies exist, you see them everywhere’ (Kreft & de Leeuw, 1998, p. 1).

Multilevel modelling (MLM) provides a technique for analysing such data. It accounts for the hierarchical structure of the data and the complexity that such structure introduces in terms of correctly modelling variability (Snijders & Bosker, 2012). Multilevel models are often referred to as hierarchical linear models, mixed models, mixed-effects models or random-effects models. Researchers often use these terms interchangeably, although there are slight differences in their meanings. For instance, hierarchical linear model is a more circumscribed term than the others: it assumes a normally distributed outcome variable. In contrast, mixed-effects or random-effects models are more general than multilevel models: they denote non-independence within a data set, but that non-independence does not necessarily need to be hierarchically nested.

In this book, we focus on one particular type of random-effects model: the multilevel model, in which units are hierarchically nested within higher level structures. Other common random-effects models include cross-classified random-effects models, which account for non-independence that is crossed, rather than nested. For example, in longitudinal educational studies, students often change teachers or transfer from one school to another; hence, students experience distinct combinations of teachers or schools. In such scenarios, students are cross-classified by two teachers or two schools. Multiple-membership models allow for membership in multiple clusters simultaneously. Although cross-classified and multiple-membership models are random-effects models, they are not purely multilevel models because they do not exhibit clean, hierarchical data structures. This book focuses on hierarchical linear modelling (HLM)/multilevel modelling (MLM). Interested readers should refer to the following resources for more information about cross-classified and multiple-membership models: Airoldi et al. (2015), Beretvas (2008) or Fielding and Goldstein (2006). In addition, this book assumes normally-distributed continuous outcomes. To learn more about using MLM techniques with non-normal (binary, ordinal or count) outcomes, see O’Connell and McCoach (2008) or Raudenbush and Bryk (2002).

In MLM, organisational models are models in which people are clustered within hierarchical structures such as companies, schools, hospitals or towns. Multilevel models also prove useful in the analysis of longitudinal data, where observations across time are nested within individuals.

In this chapter, we introduce the MLM framework and discuss two-level multilevel models in which people are clustered within organisations or groups. We begin our introduction to MLM by introducing basic terms and ideas of MLM as well as introducing one of the most fundamental concepts in the analysis of clustered data: the intra-class correlation coefficient (ICC). Subsequently, Chapter 2 provides an overview of standard two-level multilevel organisational models, and Chapter 3 illustrates fundamental MLM techniques with an applied example. In Chapters 4–6, we turn our attention to structural equation modelling. In Chapters 7 and 8, we return to MLM, demonstrating its application to individual growth models.

Nested data and non-independence

Most traditional statistical analyses assume that observations are independent of each other. In other words, the assumption of independence means that subjects’ responses are not correlated with each other. For example, imagine that a survey company administers a survey to a sample of participants. Under the assumption of independence, one participant’s responses do not correlate with the responses of any of the other participants. The assumption of independence might be reasonable when data are randomly sampled from a large population. However, the responses of people clustered within naturally occurring organisational units (e.g. schools, classrooms, hospitals, companies) are likely to exhibit some degree of relatedness, given that they were sampled from the same organisational unit. For instance, students who receive instruction together in the same classroom, delivered by the same teacher, tend to be more similar in their achievement (and other educational outcomes) than students instructed by different teachers.

Observations within a given cluster often exhibit some degree of dependence (or interdependency). In such a scenario, violating the assumption of independence produces incorrect standard errors that are smaller than they should be. Therefore, inferential statistical tests that violate the assumption of independence have inflated Type I error rates: they produce statistically significant effects more often than they should. The Type I error rate is the probability of rejecting the null hypothesis when the null hypothesis is correct. Alpha, the desired/assumed Type I error rate, is commonly set at .05. However, alpha may not equal the actual Type I error rate if we fail to meet the assumptions of our statistical test (i.e. normality, independence, homoscedasticity etc.). MLM techniques allow researchers to model the relatedness of observations within clusters explicitly. As a result, the standard errors from multilevel analyses account for the clustered nature of the data, resulting in more accurate Type I error rates.

The advantages of MLM are not purely statistical. Substantively, it may be of great interest to understand the degree to which people from the same cluster are similar to each other and to identify variables that help predict variability both within and across clusters. Multilevel analyses allow us to exploit the information contained in clustered samples and to partition the variance in the outcome variable into between-cluster variance and within-cluster variance. We can also use predictors at both the individual level (level 1) and the group level (level 2) to try to explain this between- and within-cluster variability in the outcome variable.

In general, MLM techniques allow researchers to model multiple levels of a hierarchy simultaneously, partition variance across the levels of analysis and examine relationships and interactions among variables that occur at multiple levels of a hierarchy. In MLM, a level is ‘a focal plane in social, psychological, or physical space that exists within a hierarchical structure’ (Gully & Phillips, 2019, p. 11). Generally, the levels of interest within an analysis depend on the phenomena and research questions (Gully & Phillips, 2019). For example, in a study of instructional techniques, where students are nested within teachers, students are level-1 units and teachers are level-2 units. In contrast, in a study of teachers’ perceptions of their principals’ leadership, teachers are nested within principals. In this case, teachers are level-1 units and principals are level-2 units. Often, researchers use the term organisational model to refer to cross-sectional MLM where individuals (level-1 units) are clustered within some sort of organisational, administrative, social or political hierarchy (level-2 units).

Traditional correlations and regression-based approaches estimate the relationship between two variables. However, standard single-level analyses (which ignore the clustered/hierarchical nature of the data) assume that the relationship between the variables is constant across the entire sample. MLM allows the relationships among key substantive variables to randomly vary across clusters. For example, the relationship between socio-economic status (SES) and achievement may vary by school. In some schools, student SES may be a strong (positive) predictor of students’ subsequent academic achievement; in other schools, SES may be completely unrelated to academic achievement (Raudenbush & Bryk, 2002).

Additionally, in MLM, researchers can study relationships among variables that occur at multiple levels of the data hierarchy as well as potential interactions among variables at multiple levels while allowing relationships among lower-level variables to randomly vary by cluster. How much of the between-cluster variability in these relationships (or in the cluster means) can be explained by cluster-level variables? For instance, imagine we want to study the relationships between student ability, teaching style and academic achievement. The data are clustered: students are nested within teachers (classrooms). For simplicity, assume that each teacher teaches only one class. Therefore, the teacher and classroom levels are synonymous, and student ability varies across different students taught by the same teacher. Consequently, student ability is an individual-level (or level-1) variable. Although teaching style varies across teachers, every student within a given teacher’s class is exposed to a single teacher with one individual teaching style. Therefore, teaching style varies across classrooms, but not within classrooms, so teaching style is a classroom/teacher (cluster) level, or level-2 variable.

Of course, the effect of a teacher’s teaching style does not necessarily have the same effect on all students. In our current example, we might hypothesise that teaching style moderates the effect of student ability on student achievement. In other words, the relationship between student ability and student achievement varies as a function of teachers’ teaching style. For example, some teachers may strive to ensure that all students in the class meet the same set of grade-level standards and are exposed to the same content at the same level, ensuring that all students in the class have the same set of skills and knowledge. In contrast, other teachers may differentiate instruction to meet the needs of individual students. We hypothesise that the relationship between ability and achievement would likely be stronger in the classrooms where teachers differentiate instruction than in the standards-based classrooms. In a standard linear regression model, we can include an interaction between teaching style and student ability. However, the multilevel framework allows the slope for the effect of students’ ability on achievement to randomly vary across classrooms, even after controlling for all teacher- and student-level variables in the model. If the ability/achievement slope randomly varies, even after including teaching style in the model, the relationship between ability and achievement does indeed vary across classrooms but the teachers’ teaching style does not fully explain the between-class variability in the ability/achievement relationship. Perhaps, other omitted classroom-level variables may explain the variability in the ability/achievement relationship across classes. MLM allows us to ask and answer more nuanced questions than are possible within traditional regression analyses.

As the preceding paragraphs highlight, multilevel models are incredibly useful for studying organisational contexts like schools, companies or families. However, many other types of data exhibit dependence. For instance, multiple observations collected on the same person represent another form of nested data. Growth curve and other longitudinal analyses can be reframed as multilevel models, in which observations across time are nested within individuals. Using the MLM framework, we partition residual or error variance into within-person residual variance and between-person residual ...

Table of contents

• Cover
• Half Title
• Acknowledgements
• Title Page
• Copyright Page
• Contents
• Illustration List
• About the Authors
• Acknowledgements
• Preface
• 1 Clustering and Dependence: Our Entry Into Multilevel Modelling
• 2 Multilevel Modelling A Conceptual Introduction
• 3 Multilevel Model Building Steps and Example
• 4 Introduction to Structural Equation Modelling
• 5 Specification and Identification of Structural Equation Models
• 6 Building Structural Equation Models
• 7 Longitudinal Growth Curve Models in MLM and SEM
• 8 An Applied Example of Growth Curve Modelling in MLM and SEM
• Appendix 1 Brief Introduction to Matrices and Matrix Algebra
• Appendix 2 Linking Path Diagrams to Structural Equations
• Appendix 3 Wright’s Standardised Tracing Rules
• Appendix 4 WRIGHT’S UNSTANDARDISED Tracing Rules and Covariance Algebra
• Glossary
• References
• Index