The Rorschach in Multimethod Forensic Assessment

Conceptual Foundations and Practical Applications

Robert E. Erard, F. Barton Evans, Robert E. Erard, F. Barton Evans

• Publisher: Routledge
• Year: 2016
• Print ISBN: 9781138925090
• eBook ISBN: 9781317404651
• Topic: Psychology
• Subtopic: Mental Health in Psychology
• Index: Psychology

This volume demonstrates how multimethod forensic assessment with the Rorschach adds incremental validity, insight, and practical value. Case discussions by leading forensic psychologists illustrate the integration of contemporary Rorschach assessment with the MMPI-2 and MMPI-2-RF, the PAI, and the HCR-20. This text addresses a wide range of forensic applications including child custody, psychological trauma, personal injury, psychotic offenders, competency evaluations, immigration cases, and impression management. It also shows how the recently developed Rorschach Performance Assessment System (R-PAS) effectively enhances the use of the Rorschach in forensic cases, while offering guidance for Comprehensive System users as well.

Part I

Conceptual Foundations

Chapter 1

Toward an Integrative Perspective on the Person

Opportunities and Challenges of Multimethod Assessment

Robert F. Bornstein

The human mental apparatus is powerful, indeed able to react to changing circumstances in a matter of milliseconds, perceive the faintest of stimuli, carry out multiple tasks simultaneously, and process vast amounts of information with minimal conscious effort. Despite the impressive power of the human mind and brain, we are surprisingly limited when we focus our attention on ourselves. Evidence confirms that we have only modest insight into many of our mental activities (Bargh & Williams, 2006; Wilson, 2009); as a result, we are not very good at reporting accurately on our thoughts, feelings, attitudes, goals, and motives. We are also poor judges of our own behavior: We cannot recall past events accurately, routinely misconstrue ongoing behavior even as we are exhibiting it, and cannot predict how we will behave in the future (Fernandez, 2013; Kahneman, 2003; Slovic et al., 2002). Complicating the situation, even in those areas where we manage to see things somewhat accurately, we do not always choose to provide accurate self-reports, for a variety of reasons (e.g., wanting to appear conscientious and adaptable to impress a potential employer, choosing to present ourselves as maladjusted and dysfunctional to abrogate some unwanted responsibility).

If humans could (and did) report accurately on their behavior and mental life, there would be no need for psychological assessment. Alas, they cannot—and do not—and as a result, psychologists have developed a broad array of assessment tools for use in laboratory, clinical, and forensic settings. These run the gamut from questionnaires and interviews to tests that require the respondent to interpret ambiguous stimuli, tell stories about pictures, copy geometric figures from cards, sort objects into categories, and provide open-ended descriptions of important people in his or her life.

Despite the availability of a broad array of assessment methods, psychologists today tend to rely primarily on self-report tests. When Ready and Veague (2014) surveyed directors of clinical psychology doctoral programs, they found that all but one of the most widely taught instruments were self-report scales, with little attention to integrating data from measures that use contrasting methods to assess a common construct (see also Camara, Nathan, & Puente, 2000; Childs & Eyde, 2002). The increasing reliance on mono-method assessment that characterizes current training in psychology is reflected in the research literature as well. For example, more than 80% of studies published in five leading personality disorder journals between 1991 and 2000 relied exclusively on self-report data, both to select participants and to assess outcome (Bornstein, 2003). During this same period 73% of empirical studies in the American Psychological Association’s (APA’s) seven most widely subscribed journals relied exclusively on self-report outcome measures (Bornstein, 2001). When Hogan and Agnello (2004) surveyed 696 research reports from the APA’s Directory of Unpublished Experimental Mental Measures, identifying the types of validity evidence reported for each, they found that for 87% of tests, the only validity evidence involved correlations between test scores and scores on self-report scales. Similar results were obtained by Bornstein (2011) in his survey of construct validity studies in five leading assessment journals. Given people’s limited self-awareness and inclination to dissimulate (i.e., to “fake good” or “fake bad”), psychology’s reliance on self-report tests is a recipe for failure in the laboratory, clinic, and courtroom.

From Psychological Testing to Multimethod Assessment

Psychologists often use the terms testing and assessment interchangeably, but in fact, they mean very different things. Meyer et al. (2001, p. 143) provided an excellent summary of the conceptual and practical differences between psychological testing and psychological assessment. They wrote:

Testing is a relatively straightforward process wherein a particular test is administered to obtain a specific score. Subsequently, a descriptive meaning can be applied to the score based on normative, nomothetic findings. In contrast, psychological assessment is concerned with the clinician who takes a variety of test scores, generally obtained from multiple test methods, and considers the data in the context of history, referral information, and observed behavior to understand the person being evaluated, to answer the referral questions, and then to communicate findings to the patient, his or her significant others, and referral sources.

As Cates (1999) wryly observed, he would be at a loss for words if an astute attorney asked him to provide evidence regarding the reliability and validity of psychological assessment results. He went on to note that “the care provided in the development of psychological tests overlooks the use of these techniques in combination in a battery” (Cates, 1999, p. 632). The Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, and National Council on Measurement in Education, 2014) does not address this question either, and as Bornstein (2015) noted, because the utility of psychological assessment is reflected in the degree to which assessment data provide clinically or forensically useful information, traditional validation methods (e.g., derivation of indices of reliability and validity) are not useful in this domain. The utility of psychological assessment results can only be evaluated with respect to the question that the assessment is intended to address. Documenting assessment utility is particularly challenging because a broad array of issues are addressed in clinical and forensic assessments. These include questions related to diagnosis, defense style, potential to benefit from treatment, treatment effectiveness/outcome, mental status, competency, parental fitness, fitness for duty, risk management, death penalty mitigation, and employment litigation (see Meyer et al. [2001] and Gacono & Evans [2008] for overviews of issues in this area).

Multimethod assessment (MMA) has several advantages over assessment that uses measures from within a single modality, especially when complex clinical and forensic questions are being addressed. Myriad studies have shown that across a broad array of domains, correlations between scores on measures that use different methods to assess the same construct are typically in the 0.10–0.30 range (Lobbestael, Cima, & Arntz, 2013; Mihura, Meyer, Dumistrascu, & Bombel, 2013; Zeigler-Hill, Fulton, & McLemore, 2012). Moreover, as the Standards for Educational and Psychological Testing (AERA et al., 2014) notes, unless it is specifically aimed at quantifying a single aspect of functioning (e.g., implicit self-esteem, self-attributed narcissism), mono-method assessment yields data that suffer from construct underrepresentation, capturing some—but not all—features of the construct it purports to measure. Simply put, mono-method assessment invariably yields an incomplete picture of the person.

A Process-Focused Classification of Psychological Tests

Because different assessment methods engage different psychological processes, they illuminate different aspects of functioning and predict different features of an individual’s underlying motives, traits, and behavioral predispositions (Bornstein, 2009, 2011). For example, studies have shown that scores derived from interviews and questionnaires best predict goal-directed behavior exhibited in structured situations, whereas data derived from the Rorschach Inkblot Method (RIM) and Thematic Apperception Test (TAT) tend to better predict spontaneous behavior exhibited in vivo (see Bornstein, 2002a, Jenkins, 2008; McClelland, Koestner, & Weinberger, 1989; McGrath, 2008). Other studies have shown that integrating self-report test data with data derived from indirect measures, such as the RIM, can illuminate underlying personality dynamics (Cogswell, Alloy, Karpinski, & Grant, 2010; Vater et al., 2013). Thus, as long as the validity of each measure within a test battery is reasonably well established, there is strong support for combining scores from diverse measures and methods to yield a more nuanced and accurate set of assessment data.

How is the psychologist to decide which measures and methods to employ in constructing an integrated multimethod battery in clinical and forensic settings? In part to address this question, Bornstein (2007, 2011) provided a preliminary process-based classification of widely used psychological tests. In an updated version of this classification, these measures may be divided into five broad categories.

Self-Report Tests

Self-report test scores reflect the degree to which the person attributes various traits, feelings, thoughts, motives, behaviors, attitudes, and experiences to him- or herself. Because they are efficient and cost-effective, self-report tests are far and away the most widely used type of measure in both research and clinical settings (Bornstein, 2011; Ready & Veague, 2014). In interpreting self-report test data, it is important to keep in mind that self-reports do not represent veridical indices of behavior and mental life, but instead reflect inferences about the self that are subject to a variety of information processing biases (e.g., the fundamental attribution error, self-serving attributional bias, the actor-observer effect; see Bornstein, 2015). The Beck Depression Inventory (Beck, Steer, & Brown, 1996), the Personality Assessment Inventory (Morey, 1990), and the NEO Personality Inventory (NEO-PI; Costa & McCrae, 1985) would all be included in this category.

Stimulus Attribution Tests

Traditionally called projective tests, and more recently performance-based tests, in stimulus attribution tests, the person attributes meaning to an ambiguous stimulus, with attributions determined in part by stimulus characteristics and in part by the person’s cognitive style, emotions, motives, and need states (see McGrath, 2008; Meyer et al., 2011; Weiner, 2004). Like the self-attributions that emerge as respondents complete self-report measures, the stimulus attributions that occur as people interpret ambiguous stimuli are shaped by a broad array of processes, including schema activation, variations in mood and anxiety level, ego defenses, characteristics of the examiner and the setting in which testing takes place, and self-presentation/impression management strategies (Bornstein, 2007; Ganellen, 2007). The RIM is the most widely used and well-known stimulus attribution test; others include the TAT (Murray, 1943) and the Holtzman (1961) Inkblot Test.

Constructive Tests

In constructive tests, generation of test responses requires the person to create or construct a novel image or written description within parameters defined by the tester. The Draw-a-Person Test (and other projective drawings) would be classified in this category, as would various open-ended self-descriptions (e.g., Blatt, Chevron, Quinlan, Schaffer, & Wein’s [1981] measure of Qualitative and Structural Dimensions of Object Representations; Bruhn’s [1992a, 1992b] Early Memories Procedure). In contrast to stimulus attribution tests, which require respondents to describe stimuli whose essential properties were determined a priori, in constructive tests, the “stimulus” exists only in the mind of the respondent (e.g., a self-representation or parental image; see Bers, Blatt, Sayward, & Johnston, 1993).

Behavioral Indices

In some behavioral tests, scores are derived from indices of a person’s behavior exhibited and measured in vivo, as in ambulatory assessment (a technique wherein researchers sample real-world behavior at randomly selected times, in multiple contexts; Trull & Ebner-Priemer, 2013). Behavior may also be examined in a controlled setting (e.g., using joystick feedback tasks wherein moment-by-moment behaviors are rated as they occur; see Pincus et al., 2014). Other behavioral tests assess the person’s unrehearsed performance on one or more structured tasks designed to tap attentional resources, working memory, and other cognitive skills (e.g., the Bender [1938] Visual-Motor Gestalt Test, the Attentional Capacity Test [Weber, 1988]).

Informant Reports

Scores on tests in this category are based on informants’ ratings or judgments of a person’s characteristic patterns of responding (e.g., the therapist version of the Shedler-Westen Assessment Procedure [Shedler & Westen, 1998], the informant-report version of the NEO-PI [Costa & McCrae, 1988]). In contrast to observational measures, which are based on direct observation of behavior in real time, informant-report tests are based on informants’ retrospective, memory-derived conclusions regarding characteristics of the target person. Like self-reports, informant reports are subject to all the biases and distortions that come into play as the informant reconstructs memories of past events.

Convergences and Divergences in Psychological Test Results

Given the contrasting psychological processes engaged by different tests, it is important to emphasize test score divergences as well as convergences to maximize the utility of MMA data in clinical and forensic settings. For example, researchers have found that patients with borderline pathology often perform quite well on measures with a high degree of structure (e.g., intelligence tests), while performing poorly on less structured instruments (Carr & Goldstein, 1981). Saltzman-Benaiah and Lalonde (2007) found that divergences between highly structured and less structured indices of social competence helped predict teacher ratings of children’s ability to mentalize (i.e., to infer the perspective of another person). With respect to differential diagnosis, Bornstein (1998) demonstrated that individuals with dependent personality disorder traits and symptoms scored high on both self-report and RIM measures of interpersonal dependency, whereas individuals with histrionic personality disorder traits and symptoms obtained high RIM dependency scores, but low self-report dependency scores. Valiente et al. (2011) documented characteristic patterns of implicit-explicit self-esteem discrepancies that distinguish patients with paranoid delusions from depressed patients and healthy controls. Along somewhat different lines, Becerra-Garcia et al. (2013) found that performance on Part A of the Trail-Making Test (TMT; Reitan, 1992) predicted Five-Factor Model (FFM) extraversion scores in a sample of male sex offenders, whereas performance on Part B of the TMT predicted FFM openness scores, in part because Parts A and B tap contrasting attentional and self-regulatory processes.

Thus triangulating across methods that engage different psychological processes, both between and within measures, is a central element of MMA (Bornstein & Hopwood, 2015; Hopwood & Bornstein, 2014). With this in mind, a number of investigators have examined the underlying dynamics of personality traits and dimensions of psychopathology by contrasting the results obtained for self-reports and other types of measures (e.g., stimulus attributions, behavioral indices) of parallel constructs (e.g., self-esteem, narcissism, psychopathy; see Lobbestael, et al., 2013; Vater et al., 2013; Zeigler-Hill et al., 2012). From an interpersonal perspective, researchers have explored dispositional and situational variations in inter- and intrapersonal dynamics by contrasting self-reports, reports by knowledgeable others, observational ratings, and ambulatory assessments of core traits (e.g., agency, communion, dominance, nurturance; see Hopwood, Wright, Ansell, & Pincus, 2013; Pincus et al., 2014).

In addition to illuminating underlying dynamics, triangulating across methods that engage different psychological processes can help the assessor understand the role that self-perception and self-presentation effects may play in shaping assessment results. A number of psychological tests include validity indices designed to quantify the patient’s approach to the test situation (e.g., certain MMPI-2 validity scale scores [see Butcher, Hass, Greene, & Nelson, 2015]; RIM task enga...

Table of contents

• Cover
• Title
• Copyright
• Dedication
• Contents
• List of Contributors
• Preface
• Acknowledgments
• About the Editors
• Part I. Conceptual Foundations
• Part II. Forensic Applications
• Part III. Special Topics
• Index