Assessing English Language Learners

Theory and Practice

Guillermo Solano Flores

• Publisher: Routledge
• Year: 2016
• ISBN:
  9781134073504
• Edition: 1
• Topic: Education
• Subtopic: Education General

Assessing English Language Learners explains and illustrates the main ideas underlying assessment as an activity intimately linked to instruction and the basic principles for developing, using, selecting, and adapting assessment instruments and strategies to assess content knowledge in English language learners (ELLs). Sensitive to the professional development needs of both in-service and pre-service mainstream teachers with ELLs in their classrooms and those receiving formal training to teach culturally and linguistically diverse students, the text is designed to engage readers in

• 
  
  

• viewing assessment as a critical part of teaching

• 
  
  

• appreciating that assessments provide teachers with valuable information about their students' learning and thinking

• 
  
  

• becoming aware of the relationship among language, culture, and testing

• 
  
  

• understanding the reasoning that guides test construction

• 
  
  

• recognizing the limitations of testing practices

• 
  
  

• being confident that assessment is an activity classroom teachers (not only accountability specialists) can perform

Highlighting alternative, multidisciplinary approaches that address linguistic and cultural diversity in testing, this text, enhanced by multiple field-tested exercises and examples of different forms of assessment, is ideal for any course covering the theory and practice of ELL assessment.

1 Understanding assessment

DOI: 10.4324/9780203521953-2

Overview

This chapter introduces the reader to concepts and ideas that are critical to understanding assessment practices, especially in relation to students who are English language learners (ELLs). While some definitions are not necessarily formal, they should allow the reader to follow the ideas discussed throughout the book. Some of the concepts discussed in this chapter are discussed in more detail in Chapter 9.

The section “Multiple Facets of Assessment” provides a comprehensive view of the concept of assessment as a process, an instrument, and a system. The section “Basic Concepts in Assessment” discusses construct, measurement error, standardization, sampling, validity, reliability, and fairness.

The word assessment is used to refer to the activity of integrating and evaluating information relevant to student academic achievement from different sources (see Cronbach, 1990). Also, it is used to refer to “a tool designed to observe students’ behavior and produce data that can be used to draw reasonable inferences about what students know” (Pellegrino, Chudowsky, & Glaser, 2001, p. 42). The terms assessment and test are used interchangeably to refer to that tool or instrument. Also, the terms item and task are used interchangeably. However, the former is used more frequently to refer to questions that require short answers from the students and the latter is used more frequently to refer to more complex activities involving constructed responses.

Multiple facets of assessment

Assessment as a process

Assessment can be viewed as a process of gathering different kinds information about student achievement which, ideally, should provide “critical information for many parts of the education system, including instructional decisions, holding schools accountable for meeting learning goals, and monitoring program effectiveness” (Wilson & Bertenthal, 2006, p. 3). Assessment comprises actions taken with the intent to determine students’ knowledge or skills according to a set of goals or expectations. It encompasses the reasoning and procedures used to generate tasks, score student responses, and transform the information on those responses into measures of academic achievement.

In Knowing What Students Know, Pellegrino, Chudowsky, and Glaser (2001) offer a simple but compelling view of assessment in terms of three components: cognition, observation, and interpretation. Cognition refers to the mental processes and activities underlying proficiency in a given knowledge domain. A good test should not be developed without having a clear idea of the cognitive processes involved in responding to it. As an example, take subtraction. To develop a test on this domain, we need to formalize our beliefs about the processes that take place when students solve different kinds of subtraction problems. Among many others, these processes may include retrieving information from memory, applying computation routines, and representing subtraction problems in multiple formats. Depending on how subtraction is defined, the processes may also include recognizing when a problem is a subtraction problem, and constructing strategies to solve problems through subtraction, among many other activities.

Observation refers to the kinds of tasks that elicit from students responses that show competence in the knowledge domain assessed. In the example of subtraction, depending on the knowledge and skills of interest, a test might include, among others, straight computation problems, word problems, and problems that contain complex contextual information. These problems need to be presented in different response formats, from those that involve information recognition and recall to those that involve open, elaborate responses.

Interpretation refers to the different patterns of student responses that correspond to varying levels of student competence. A test would be useless without a set of rules for determining degrees of correctness of the possible student responses. In the simplest case, we want to know which option is the correct response for each of the items in a multiple-choice test. For constructed-response tasks, there should be a set of rules that allow scorers to make decisions about the quality of the process involved in the student’s response (i.e. the assumed cognitive activity that takes place during problem solving) and the product of that response (i.e. the outcome of that cognitive activity).

The process of assessment can be examined according to the contexts in which assessment takes place, the functions assessment serves, and the uses of assessments (Figure 1.1). Two assessment contexts are identified, classroom and large-scale.

Figure 1.1 Contexts, functions, and uses in the assessment process.

Classroom assessment comprises the formal and informal assessment activities that take place in the classroom (e.g. quizzes, classroom conversations, assignments) developed, used, or selected by the teacher according to the teaching context, the instructional goals, and the teacher’s knowledge of their students on aspects such as the students’ progress, learning styles, or strengths and weaknesses. Formative assessment is embedded in instruction and takes place, for example, as a part of an instructional unit, lesson, or course; teachers use it to inform their teaching and to provide feedback to students (Black & Wiliam, 2009). In contrast, summative classroom assessment takes place at the end of that instructional unit, lesson, or course; teachers use it to grade their students and to serve reporting purposes.

Large-scale assessment comprises the formal assessment activities performed by entities external to the classroom, such as a state’s department of education; it involves the testing of large populations of students. Diagnostic assessment is not embedded in instruction and may take place at the beginning of a school year with the purpose of determining, for example, the students’ proficiency in English for instructional placement decisions (e.g. bilingual, transitional, or English-only programs) or for classification decisions (e.g. to determine if it is appropriate to assume that students are proficient enough in English to be tested in that language). Summative large-scale assessment informs high-stake and accountability decisions such as admission, retention, or graduation for individual students.

Assessment as an instrument

While everybody has an intuitive idea of what assessments are, a definition is needed to be able to discuss the complexities of the field of assessment:

An assessment or test can be defined as an instrument composed by a set of tasks and their accompanying response formats, scoring systems, and administration procedures, designed to obtain information on the extent to which an individual or a population of examinees possess the knowledge or skills that are relevant to a given domain.

The term task is used to refer generically to any form of item, question, or problem included in a test. Tasks vary tremendously on the amount and complexity of information provided to students (e.g. directions, contextual information, data, diagrams), the knowledge and skills they are intended to address (e.g. recognizing information, evaluating, applying a procedure, solving a problem), and the kinds of activities involved in their completion (e.g. performing computations, reading a passage, writing a report, conducting experiments with various pieces of equipment).

Response format is the means by which students’ responses are captured. The range of response formats is wide, from multiple-choice and fill-in-the-blank items—which restrict the variety of responses required from the student—to essays, logs, notebooks, concept maps, portfolios, and real-time observation—which lend themselves to a wide variety of responses varying in length and form.

It is well known that different types of tasks and response formats are sensitive to different aspects of knowledge. This is due to the fact that they pose different sets of cognitive demands (see Baxter, Shavelson, Goldman, & Pine, 1992; Martinez, 1999). To a large extent, this is due also to the fact that different tasks and response formants pose different sets of linguistic demands. For example, while multiple-choice items require virtually no writing and contain relatively fewer and shorter sentences than open-ended items, their grammatical structures tend to be more complex. Because of this trade-off between challenges and affordances, a simple way of addressing the challenges of validly assessing ELL students in English consists of using different types of tasks and response formats. This approach is intended to ensure that these students are exposed to different sets of linguistic challenges and affordances, thus providing them with different sets of opportunities to demonstrate their knowledge.

A scoring system is the set of rules based on which a score is assigned to a student’s response or to one aspect of the student’s response. The simplest scoring system is that of multiple-choice and certain fill-in-the-blank items in which, whether the response is correct or incorrect can be told unambiguously from the option selected or the word or short phrase written by the student. In contrast, more complex tasks and response formats (e.g. open-ended items, constructed-response tasks) involve human judgment and make it necessary the use of scoring rubrics—documents which describe the characteristics of student responses at various levels of correctness (see Arter & McTighe, 2000). Holistic rubrics focus on the overall characteristics of performance; they consist of descriptions of patterns with which student performance is observed. Analytic rubrics focus on the presence or absence of certain characteristics separately, regardless of how these characteristics are related.

An important challenge in the design of scoring rubrics consists of ensuring that they do not favor certain writing styles over others. This is especially important in the testing of young students from linguistic and cultural minorities. Cultures differ as to how they encourage or discourage long and short utterances among children, especially when they talk to adults (see Heath, 1983). This can be reflected in the length or degree of elaboration of written responses given by students to open-ended questions. The scoring rubrics should prevent raters from unduly valuing long responses or responses given with certain writing styles over short responses.

An administration procedure specifies how tasks should be given to examinees. The amount of time students are given to complete a test, whether they should complete tasks individually or in pairs, and how any materials used in the test should be distributed to students and collected from them are examples of the aspects of testing that an administration procedure should specify.

While the administration procedure is not always considered as a fundamental component of tests, its importance is more evident in the testing of ELL students and the use of testing accommodations—modifications on the way in which tests are administered with the purpose of minimizing the effect of ELLs’ limited proficiency in the language of testing (Abedi, Hofstetter, & Lord, 2004). Providing certain testing accommodations to ELL students often requires the alteration of the administration procedure. The most obvious case is the accommodation consisting of giving these students more time to complete their tests.

Assessment as a system

The term assessment system refers to the coordinated work of different organizations (e.g. schools, school districts, departments of education) with the purpose of obtaining and reporting information on their students’ academic achievement (see Wilson & Bertenthal, 2006). Typically, the content of instruments generated by assessment systems is aligned to normative documents such as standards documents. Among different types of standards documents are content standards, which prescribe what students should know and be able to do by grade and content area (Raizen, 1997).

State assessment systems test students of all grades every year on each of several areas such as science, mathematics, and reading. Other assessment systems test students of selected grades in cycles that are longer than a year. That is the case of the National Assessment of Educational Progress (NAEP), whic...