Introduction to the Taxometric Method

A Practical Guide

John Ruscio, Nick Haslam, Ayelet Meron Ruscio

• Publisher: Routledge
• Year: 2013
• Print ISBN:
  9780805859768
• eBook ISBN:
  9781135809959
• Topic: Psychology
• Subtopic: History & Theory in Psychology
• Index: Psychology

Introduction to the Taxometric Method is a user-friendly, practical guide to taxometric research. Drawing from both classic and contemporary research, it provides a comprehensive introduction to the method. With helpful tools and guidance, the book is intended to teach those new to the method, as well as those already familiar with it, tips on how to conduct and evaluate taxometric investigations. The book covers a broad range of analytic techniques, describing their logic and implementation as well as what is known about their performance from systematic study.

The book opens with the background material essential to understanding the research problems that the taxometric method addresses. The authors then explain the data requirements of taxometric analysis, the logic of each procedure, factors that can influence results and lead to misinterpretations, suggestions for choosing the best procedures, and methodological safeguards to prevent erroneous conclusions. Illustrative examples of each procedure and consistency test demonstrate how to perform analyses and interpret results using a variety of data sets. A checklist of conceptual and methodological issues that should be addressed in any investigation is included. The downloadable resources provide a variety of programs for performing taxometric analyses along with simulations and analyses of data sets.

Introduction to the Taxometric Method is ideal for researchers and students conducting or evaluating taxometric studies in the social and behavioral sciences, especially those in clinical and personality psychology, as well as those in the physical sciences, education, biology, and beyond. The book also serves as a text for courses on this method, or as a supplement in psychological assessment, statistics, or research methods courses.Familiarity with taxometrics is not assumed.

PART TWO

TAXOMETRIC METHOD

CHAPTER FOUR

Data Requirements for Taxometrics

The previous chapter provided an overview of several approaches to evaluating latent structure, with particular emphasis on the taxometric method. Before introducing the procedures included in the taxometric method, however, it is important to consider what sort of data one must have to perform these procedures. Like other data-analytic tools, taxometric procedures require adequate data to yield informative results. Several features of a data set influence its appropriateness for taxometric analysis, including the nature and construction of the sample, the breadth and appropriateness of the indicator set, and important statistical characteristics of the data. We review the relevant conceptual and empirical data considerations, including the conventional guidelines derived from Monte Carlo studies that have been used to judge the adequacy of a data set for taxometric investigation.

Following this discussion, we introduce a technique that is intended to supplement these more general guidelines with direct empirical evaluation of the adequacy of a particular data set for a planned taxometric analysis. The technique involves the generation of empirical sampling distributions of results through analyses of simulated taxonic and dimensional comparison data that reproduce key characteristics of the research data (e.g., sample size, indicator distributions and correlations). These comparison data are generated using a bootstrap procedure in which databased estimates of population parameters are used to draw random samples of data (Efron & Tibshirani, 1993). This is done first in a way that yields comparison data that conform to the taxonic structural model and then in a way that yields comparison data that conform to the dimensional structural model. All bootstrap samples of comparison data are then submitted to parallel taxometric analyses, yielding empirical sampling distributions of taxometric results representing expected patterns for each structure. To the extent that these distributions diverge, one gains confidence that the unique set of research data is likely to afford a genuine test between these two competing structural hypotheses. After describing the rationale behind this approach and highlighting the strengths and limitations of the technique, we discuss the nuts and bolts of how it is implemented and how its results can be used to inform a taxometric investigation.

SAMPLING CONSIDERATIONS

Sample Size

On the basis of Monte Carlo research (e.g., Meehl & Yonce, 1994, 1996), Meehl (1995a) recommended that data sets submitted to taxometric analyses include a minimum of 300 cases. Although taxometric procedures have been shown to successfully distinguish taxonic from dimensional structure in smaller samples, this has generally been found only when the data possess other highly favorable characteristics (e.g., equal-sized groups separated by a large amount on indicators that are uncorrelated within groups and normally distributed along continuous scales). Because actual research data are unlikely to possess all of these desirable properties, many researchers have adopted Meehl's rule of thumb as a reasonable standard.

Haslam and Kim (2002) reviewed 66 published and unpublished taxometric studies and found a median sample size of 585. This figure jumped to 809 among the 57 published taxometric investigations in the updated review we present in chapter 10. Moreover, sample size appears to be on the rise in published taxometric reports: The median sample size was 639 for the 21 studies prior to 2000 and 923 for the 36 studies published during or after 2000. As is apparent from these trends, large sample sizes are the norm—and the expectation—in taxometric investigations.

Size and Base Rate of the Taxon

To be appropriate for taxometric analysis, a sample must not only be large, but also contain a sufficient number of putative taxon members to permit their detection in the analysis. This characteristic of the data is conventionally discussed as the taxon base rate because the taxon is often the smaller of the two groups in the sample. However, this data requirement pertains more generally to the relative sizes of the two latent groups, indicating that neither the taxon base rate (P) nor the complement base rate (Q) should be so small that the distinction between these groups is missed by the taxometric analyses. The same Monte Carlo studies cited by Meehl (1995a) to support his recommended minimal sample size found that taxometric procedures correctly identified taxonic structure in samples with a taxon base rate as low as P = .10. Although this finding was based on a small number of data configurations and involved no analyses with P < .10, many researchers have adopted the P ≥ .10 threshold as a reasonable base rate standard. However, there are several reasons to be flexible in the use of this guideline.

First, Monte Carlo studies seldom evaluated the performance of taxometric procedures in samples with taxon base rates smaller than .10. As a result, the lower limit of what these procedures can detect is presently unknown. Beauchaine and Beauchaine (2002) generated and analyzed taxonic data with base rates lower than .10 and suggested that these small taxa could be detected under some conditions, but their criterion for success was based on two consistency tests later shown to poorly identify taxonic structure (see chapter 7 for details). In another simulation study, J. Ruscio (2005) found that taxon base rates as low as .05 were sometimes estimated accurately. This finding provides only indirect evidence, but the ability to accurately estimate their size suggests that the procedures can detect small taxa under at least some data conditions.

Second, preliminary evidence suggests that the absolute number of taxon members in a sample may be at least as important as the proportion of the sample that they comprise. J. Ruscio and Ruscio (2004a) found that taxometric procedures continued to detect a taxon of a constant absolute size even when large numbers of complement members were added to the sample, causing the taxon base rate to fall well below .10. For this reason, we prefer to break with convention by referring to a small taxon rather than a low base rate taxon, underscoring the potential importance of the absolute size of the group as well as its base rate in the sample. Indeed, although the conventional rule of thumb of P ≥ .10 may be appropriate in relatively small samples, it may be somewhat conservative in especially large samples, wherein even a very low base rate may correspond to a substantial number of taxon members that can be detected by taxometric procedures.

At present, it is not well understood how the taxon base rate and the absolute size of the taxon jointly influence the sensitivity of each taxometric procedure to detecting a taxonic boundary. Thus, we suggest that researchers consider both of these values when evaluating the appropriateness of their data for a taxometric analysis. For example, if the base rate of a putative taxon is expected to be near .10, one could look to the absolute size of this taxon for additional guidance on the likely capacity of the analysis to distinguish taxonic from dimensional structure. If the taxon base rate is estimated to be lower than .10, it would be especially important to demonstrate, before proceeding with the analysis, that there are enough members of the taxon in the sample to give credence to their detection. Although no additional general guidelines are presently available to help researchers make this judgment, we discuss an approach later in this chapter that researchers can use to empirically evaluate the adequacy of a data set (including the size of the taxon) for the intended taxometric analyses.

Third, the sensitivity of taxometric procedures to the existence of a small taxon almost certainly depends on many other characteristics of the data. The validity with which the indicators separate the groups is probably most important, but also relevant are factors such as the magnitude of within-group correlations and the degree of within-group indicator skew. When all or many of these data properties are strongly favorable for taxometrics, it may be easier to detect relatively small taxa. To the extent that these data properties are weak or questionable, a small taxon may be missed. In addition, sensitivity to a small taxon may vary across taxometric procedures. A recent demonstration using one data configuration found a substantial difference in the ability of three taxometric procedures to detect an increasingly small taxon (J. Ruscio & Ruscio, 2004a).

For all of these reasons, it may be misleading to set a single acceptable base rate threshold without regard for other characteristics of the data or the analysis plan. Until further Monte Carlo research is conducted to explore this issue, we recommend that researchers (a) use the P ≥ .10 rule of thumb flexibly, (b) strive to collect data from a mixed-group population in which the putative taxon naturally occurs with a base rate closer to the ideal of .50, and (c) empirically evaluate the appropriateness of their data for each planned analysis as a more direct check of the adequate representation of taxon members.

The Population Sampled

The nature of the (mixed-group) population from which cases are sampled will influence the likely size of the taxon and the range of scores represented on the indicator variables. For example, samples drawn from relevant clinical populations will often contain a larger psychopathology taxon (as well as more intermediate or subthreshold cases) than will samples drawn from community or analogue populations, meaning that a far larger nonclinical sample will be required to amass the same number of taxon members found in a smaller but well-chosen clinical sample. In addition, the broader range of symptom severity present in an appropriate clinical sample will often result in a larger range of scores on the indicator variables, increasing the likelihood of detecting a taxon located at the upper end of the score distribution and facilitating the implementation of taxometric procedures.

It should be noted, however, that there may also be potential disadvantages to conducting taxometric investigations of psychopathology constructs using certain clinical samples. First, some clinical samples may contain too many cases belonging to the taxon. For instance, a clinic that specializes in the treatment of one particular disorder may have a client roster consisting almost entirely of individuals with that disorder, thereby including too few members of the complement to distinguish it from the pathological taxon. Similarly, if the demand for services exceeds the capacity of a particular clinic, the staff may choose to provide services only to those individuals exhibiting the most severe levels of distress or impairment. This may eliminate most or all members of the complement and artificially constrain the range of functioning within the sample to the point where the data are no longer appropriate for taxometric analysis. Second, just because a sample is drawn from a clinical population does not necessarily mean that it contains a high enough rate of a particular form of psychopathology to be studied using taxometric analysis. For example, some conditions may be too rare to be powerfully investigated in a general outpatient sample, requiring data to be collected in an inpatient facility or a specialty clinic to yield a large enough taxon for analysis. Other conditions or related constructs may be sufficiently prevalent in the general population to be appropriately studied in epidemiological samples (see Kessler, 2002a). In extremely large epidemiological samples, even members of rare taxa may be represented in sufficiently large numbers to afford the detection of taxonic structure.

Questionable Sampling Techniques

In addition to the sample considerations discussed thus far, three sampling approaches are worthy of special note because they can undermine the results of a taxometric analysis by introducing plausible alternative explanations for the observed results. One practice that has been used with some frequency is to combine patient and nonpatient samples (or other distinct samples) into a single sample for analysis. This technique is often based on the reasoning that admixing these anticipated taxon (patient) and complement (nonpatient) members allows the investigator to influence the taxon base rate, assembling a sample such that the taxon base rate will more closely approximate the ideal of .50 than might be possible in either subsample alone. There are two potential problems with this approach. First, depending on the nature of samples that are mixed together, this may systematically omit cases with intermediate or subc...

Table of contents

• Cover
• Half Title
• Full Title
• Copyright
• Contents
• Preface
• I INTRODUCTION AND BACKGROUND
• II TAXOMETRIC METHOD
• III APPLICATIONS AND FUTURE DIRECTIONS
• Appendix A: Simulating Taxonic and Dimensional Comparison Data
• Appendix B: Estimating Latent Parameters and Classifying Cases Using MAXCOV
• Appendix C: Estimating the Taxon Base Rate Using MAXEIG
• References
• Author Index
• Subject Index