Handbook of Psychology, Assessment Psychology

Irving B. Weiner, John R. Graham, Jack A. Naglieri

• Publisher: Wiley
• Year: 2012
• Print ISBN:
  9780470891278
• eBook ISBN:
  9781118282045
• Edition: 2
• Topic: Psychologie
• Subtopic: Geschichte & Theorie in der Psychologie

Psychology is of interest to academics from many fields, as well as to the thousands of academic and clinical psychologists and general public who can't help but be interested in learning more about why humans think and behave as they do. This award-winning twelve-volume reference covers every aspect of the ever-fascinating discipline of psychology and represents the most current knowledge in the field. This ten-year revision now covers discoveries based in neuroscience, clinical psychology's new interest in evidence-based practice and mindfulness, and new findings in social, developmental, and forensic psychology.

Part I

Assessment Issues

Chapter 1

The Assessment Process

Irving B. Weiner

Collecting Assessment Information

Interpreting Assessment Information

Using Assessment Information

References

Assessment psychology is the field of behavioral science concerned with methods of identifying similarities and differences among people in their psychological characteristics and capacities. As such, psychological assessment comprises a variety of procedures that are employed in diverse ways to achieve numerous purposes. Assessment has sometimes been equated with testing, but there is more to the assessment process than giving tests. Psychological assessment involves integrating information gleaned not only from test protocols but also from interviews, behavioral observations, collateral reports, and historical documents. In this regard, the Standards for Educational and Psychological Testing (American Educational Research Association [AERA], American Psychological Association, & National Council on Measurement in Education, 1999) specify that

the use of tests provides one method of collecting information within the larger framework of a psychological assessment of an individual.… A psychological assessment is a comprehensive examination undertaken to answer specific questions about a client's psychological functioning during a particular time interval or to predict a client's psychological functioning in the future. (p. 119)

The diverse ways in which assessment procedures are employed include many alternative approaches to obtaining and combining information from different sources, and the numerous purposes that assessment serves arise in response to a broad range of referral questions raised in clinical, educational, health care, forensic, and organizational settings. Chapters in this volume elaborate the diversity of assessment procedures, the nature of the assessment questions that arise in various settings, and the types of assessment methods commonly employed to address these questions.

This introductory chapter sets the stage for what is to follow by conceptualizing assessment as a three-stage process of information input, information evaluation, and information output. Information input involves collecting assessment data of appropriate kinds and in sufficient amounts to address referral questions in meaningful and useful ways. Information evaluation consists of interpreting assessment data in a manner that provides accurate descriptions of respondents' psychological characteristics and behavioral tendencies. Information output calls for using descriptions of respondents to formulate conclusions and recommendations that help to answer referral questions. Each of these phases of the assessment process requires assessors to accomplish some distinctive tasks, and each involves choices and decisions that touch on critical issues in conducting psychological assessments.

Collecting Assessment Information

The process of collecting assessment information begins with a formulation of the purposes that the assessment is intended to serve. Knowing why an assessment is being conducted helps examiners identify tests and other sources of information that are likely to provide a basis for relevant conclusions and useful recommendations. Also helpful in planning the data collection process is attention to examiner, respondent, and data management characteristics that influence the nature and utility of whatever findings are obtained.

Formulating Goals

Psychological assessments are instigated by referrals that pose questions about aspects of a person's psychological functioning or likely future behavior. Clearly stated and psychologically relevant referral questions help psychologists determine what kinds of assessment data to collect, what considerations to address in examining these data, and what implications of their findings to emphasize in their reports. Referral questions that lack clarity or psychological relevance require some reformulation to give direction to the assessment process. For example, a referral in a clinical setting that asks vaguely for personality evaluation or differential diagnosis needs to be made more specific in consultation between the psychologist and the referring person to identify why a personality evaluation is being sought or what diagnostic possibilities are at issue. Assessment in the absence of a specific referral question can be a sterile exercise in which neither the data collection process nor the psychologist's inferences can be focused in a meaningful way.

Even when referral questions are adequately specified, they may not be psychological in nature. Assessors doing forensic work are often asked to evaluate whether a criminal defendant was insane at the time of an alleged offense; however, sanity is a legal term, not a psychological term. No assessment methods are designed to identify “insanity,” nor are there any research studies in which “being insane” has been used as an independent variable. To help assessors plan their procedures and frame their reports in such instances, the referral must be translated into psychological terms, as in defining insanity as the inability to distinguish reality from fantasy.

As a further challenge in formulating assessment goals, referral questions that are specific and framed in psychological terms may still be unclear if they concern complex and multidetermined patterns of behavior. In employment evaluations, for example, a referring person may want to know which of three individuals is likely to perform best in a position of leadership or executive responsibility. To address this type of question adequately, assessors must first be able to identify psychological characteristics that are likely to make a difference in the particular circumstances, as by proceeding, in this example, with the presumption that being energetic, decisive, assertive, self-confident, and relatively unflappable is likely to contribute to effective leadership. The data collection process can then be planned to measure these characteristics, and the psychologist's eventual report can specify their presence or absence as the basis for whatever hiring decision is recommended.

Selecting Tests

The multiple sources of assessment information mentioned include:

• the results of formal psychological testing with standardized instruments;
• responses to questions asked in structured and unstructured interviews;
• observations of behavior in various types of contrived situations and natural settings;
• reports from relatives, friends, employers, and other collateral persons concerning an individual's previous life history and current characteristics and behavioral tendencies; and
• documents such as medical records, school records, and written reports of earlier assessments.

Individual assessments vary considerably in the availability and utility of these diverse sources of information. Assessments may at times be based solely on record reviews and collateral reports, because the person being assessed is unwilling to be seen by an examiner or is for some reason prevented from doing so. Some people being assessed are quite forthcoming when interviewed but are reluctant to be tested; others find it difficult to talk about themselves but are quite responsive to testing procedures; and in still other cases, in which both interview and test data are ample, there may be a dearth of collateral input or documented information on which to draw.

There is little way to know before the fact which sources of information will prove most critical or valuable in an assessment process. What collateral informants say about a person may sometimes be more revealing and reliable than what the person says about him- or herself, and in some instances historical documents may prove more informative and dependable than either first-person or collateral reports. Behavioral observations and interview data may on occasion contribute more to an assessment than standardized tests or may even render testing superfluous, whereas in other instances formal psychological testing may reveal vital diagnostic information that would otherwise not have been uncovered.

The fact that psychological assessment can proceed effectively without psychological testing helps to distinguish between these two activities. As previously noted, the terms psychological assessment and psychological testing are sometimes used synonymously, but psychological testing is only one of several sources of information typically utilized in the assessment process. (See Matarazzo, 1990; Meyer et al., 2001.) Testing, however, is the most complex and specialized of the data collection procedures used in psychological assessment, starting with the selection of an appropriate test battery from among an extensive array of available measuring instruments. (See Geisinger, Spies, Carlson, & Plake, 2007; see also Ben-Porath; Lowman and Carson; O'Brien and Young; Otero, Podell, DeFina and Goldberg; Viglione and Rivera; and Wasserman, this volume). The chief considerations that should determine the composition of a test battery are:

• the psychometric adequacy of the measures being considered;
• the relevance of these measures to the referral questions being addressed;
• the likelihood that these measures will contribute incremental validity to the decision-making process; and
• the additive, confirmatory, and complementary functions that individual measures are likely to serve when used jointly.

Psychometric Adequacy

As elaborated in the Standards for Educational and Psychological Testing (AERA et al., 1999) and in Wasserman and Bracken, this volume, the psychometric adequacy of an assessment instrument consists of the extent to which it

• involves standardized test materials and administration procedures;
• can be coded with reasonably good interscorer agreement;
• demonstrates acceptable reliability;
• has generated relevant normative data; and
• shows valid corollaries that serve the purposes for which it is intended.

Assessment psychologists may at times choose to use tests with uncertain psychometric properties, perhaps for exploratory purposes or for comparison with a previous examination in which these tests have been used. Generally speaking, however, formal testing as part of a psychological assessment should be limited to standardized, reliable, and valid instruments for which there are adequate normative data.

Relevance

The tests selected for inclusion in an assessment battery should provide information relevant to answering the questions that have been raised about the person being examined. Questions that relate to personality functions (e.g., “What kind of approach in psychotherapy is likely to be helpful to this person?”) call for personality tests. Questions that relate to educational issues (e.g., “Does this student have a learning disability?”) call for measures of intellectual abilities and academic aptitude and achievement. Questions that relate to neuropsychological functions (e.g., “Are there indications of memory loss?”) call for measures of cognitive functioning, with special emphasis on measures of capacities for learning and recall.

These examples of relevance may seem too obvious to have warranted mention. However, they reflect an important and sometimes overlooked guiding principle in psychological assessment, namely, that test selection should be justifiable for each measure included in an assessment battery. Insufficient attention to justifying the use of particular measures in specific instances can result in two ill-advised assessment practices: (1) conducting examinations with a fixed and unvarying battery of measures regardless of what questions are being asked in the individual case, and (2) using favorite instruments at every opportunity even when they are unlikely to serve any central or unique purpose in a particular assessment.

The administration of minimally useful tests that have little relevance to the referral question is a wasteful procedure that can evoke deserved criticism of assessment psychologists and the assessment process. Likewise, the propriety of charging fees for unnecessary procedures can rightfully be challenged by persons receiving or paying for these services, and the competence of assessors who give tests that make little contribution to answering the questions at issue can be challenged in such public forums as the courtroom. (See Weiner, 2009.)

Incremental Validity

Incremental validity in psychological assessment refers to the extent to which additional information increases the accuracy of a classification or prediction above and beyond the accuracy achieved by information already available. Assessors pay adequate attention to incremental validity by collecting the amount and kinds of information they need to answer a referral question, but no more than that. In theory, then, familiarity with the incremental validity of various measures when used for certain purposes, combined with test selection based on this information, minimizes redundancy in psychological assessment and satisfies both professional and scientific requirements for justifiable test selection.

In practice, however, strict adherence to incremental validity guidelines often proves difficult and even disadvantageous to implement. As already noted, it is difficult to anticipate which sources of information will prove to be most useful in an assessment. Similarly, with respect to which instruments to include in a test battery, there is little way to know whether the tests administered have yielded enough data and which tests have contributed most to understanding the person being examined, until after the data have been collected and analyzed.

In most practice settings, it is reasonable to conduct an interview and review previous records as a basis for deciding whether formal testing would be likely to help answer a referral question—that is, whether it will show enough incremental validity to warrant its cost in time and money. Likewise, reviewing a set of test data can provide a basis for determining what kind of additional testing might be worthwhile. However, it is rarely appropriate to administer only one test at a time, to choose each subsequent test on the basis of the preceding one, and to schedule a further testing session for each additional test administration. For this reason, responsible psychological assessment usually consists of one or two testing sessions comprising a battery of tests selected to serve specific additive, confirmatory, and complementary functions.

Additive, Confirmatory, and Complementary Functions of Tests

Some referral questions require selection of multiple tests to identify relatively distinct and independent aspects of a person's psychological functioning. For example, students receiving low grades may be referred for an evaluation to help determine whether their poor academic performance is due primarily to limited intelligence or to personality characteristics that are fostering negative attitudes toward achieving in school. A proper test battery in such a case would include some measure of intelligence and ...

Table of contents

• Cover
• Title Page
• Copyright
• Dedication
• Editorial Board
• Handbook of Psychology Preface
• Volume Preface
• Contributors
• Part I: Assessment Issues
• Part II: Assessment Settings
• Part III: Assessment Methods
• Author Index
• Subject Index