How to Design Questions and Tasks to Assess Student Thinking

Susan M. Brookhart

• Publisher: ASCD
• Year: 2014
• ISBN:
  9781416619277
• Topic: Bildung
• Subtopic: Evaluierung & Bewertung im Bildungswesen

With new standards emphasizing higher-order thinking skills, students will have to demonstrate their ability to do far more than simply remember facts and procedures. But what's the best way for teachers to ensure that students have such skills? In this highly accessible guide, author Susan M. Brookhart shows how to do just that, by providing specific guidelines for designing targeted questions and tasks that align with standards and assess students' ability to think at higher levels.

Aided by dozens of examples across grade levels and subject areas, readers will learn how to

* Take a student perspective and view assessment questions and tasks as "problems to solve."
* Design multiple-choice questions that require higher-order thinking.
* Understand the difference between "open" and "closed" questions and how to use open questions effectively.
* Vary and control the features of performance assessment tasks, including cognitive level and difficulty, to target different thinking skills.
* Manage the assessment of higher-order thinking within the larger context of teaching and learning.

Brookhart also provides an "idea bank" that teachers can use to jump-start their own thinking as they create assessments.

Timely and practical, How to Design Questions and Tasks to Assess Student Thinking is essential reading for 21st century teachers who want their students to excel in the classroom and beyond.

Note: This product listing is for the reflowable (ePub) version of the book.

---

Chapter 1

---

Assessing Higher-Order Thinking: Five Ws and an H

. . . . . . . . . . . . . . . . . . . .

In Liam's 5th grade science class, students memorize the properties of solids, liquids, and gases. They take quizzes with questions such as "The temperature at which a solid changes to a liquid is called the ____________." They answer questions such as "What happens when water boils?" by finding the appropriate explanations in their textbook and rewriting them in their own words to show they understand.

Fifth grader Olivia's science class uses the same textbook. The students take some quizzes, too. Most of their questions, however, are more like this: "Water boils at a lower temperature in the mountains than it does at sea level. Sam lives in the mountains. He is boiling carrots to serve at dinner. Would it take a longer or shorter time to cook the carrots than it would take at sea level? Explain your answer."

Both of these 5th graders are learning about states of matter. But Olivia will learn more and remember it longer than Liam. The kinds of questions her teacher asks help her use her knowledge of states of matter, connecting it to other knowledge and reasoning processes, and at the same time show her that there is a purpose for knowing these things.

WHO Is Likely to Benefit from Assessment of Higher-Order Thinking?

The Liams and Olivias of the world are the future, and they will need to learn to think. The world is changing quickly, and students will need to develop higher-order thinking skills and use them throughout their lives. If you don't already believe that students will truly benefit from assessment of higher-order thinking, I hope this chapter convinces you. If you do already believe it, this chapter will give you some important foundational ideas, in the form of five Ws (including this first one, "who") and an H.

WHAT Does Assessment of Higher-Order Thinking Look Like?

Alexander and her colleagues (2011) propose a definition of higher-order thinking that will serve well for this book:

Higher-order thinking is the mental engagement with ideas, objects, and situations in an analogical, elaborative, inductive, deductive, and otherwise transformational manner that is indicative of an orientation toward knowing as a complex, effortful, generative, evidence-seeking, and reflective enterprise. (p. 53)

Two big ideas stand out in this definition. One, higher-order thinking happens when students engage with what they know in such a way as to transform it. That is, this kind of thinking doesn't just reproduce the same knowledge; it results in something new. A student who analyzes a poem, for example, doesn't just recite the poem or copy it into her notebook. She identifies literary elements and techniques used in the poem and creates an interpretation based on those elements and techniques. The poem as a whole is transformed into parts and then re-formed into an interpretation. Something new is created. Important for this book is the point that the teacher would look at that creation as evidence of the thinking that went on inside the student's head.

The second big idea in this definition is the conception of knowledge as an enterprise that is "complex, effortful, generative, evidence-seeking, and reflective." Higher-order thinking only makes sense if to truly "know" something means that you can use it and transform it. In my 2010 book How to Assess Higher-Order Thinking Skills in Your Classroom, I summarized three ways in which teachers have conventionally thought about how students use knowledge: transfer, critical thinking, and problem solving. These three ways of thinking about using knowledge are not mutually exclusive. They all have in common that students apply what they know to an idea, an object, or a situation—as the definition says—and transform it into something new.

Higher-order thinking is often associated with the work of Bloom, Engelhart, Furst, Hill, and Krathwohl (1956). In fact, Bloom and his colleagues tried to avoid the idea that analysis, synthesis, and evaluation were somehow "above" knowledge, comprehension, and application in a hierarchical sense. However, the idea of helping students develop skills at using the knowledge they gain remains a value for educators, and turning to Bloom's or some other taxonomy is a common and practical way to do that. In this book, I refer to "Bloom's taxonomy" mostly in terms of the cognitive-process dimension of the revised Bloom's taxonomy (Anderson & Krathwohl, 2001): Remember, Understand, Apply, Analyze, Evaluate, Create.

Both tests and performance assessments can tap higher-order thinking. In this book, I use the word test to mean a paper-and-pencil or computer-based set of questions that students must answer, typically in a relatively short, fixed time period and in a supervised setting. The questions that are on tests are sometimes called test items, but in this book I mostly just call them questions.

A performance assessment has two parts: a task for students to perform and a scoring scheme to judge the performance. Performance assessment tasks can require students to demonstrate a process (e.g., sing a song, recite a Shakespeare monologue, use safety equipment properly) or produce a product (e.g., a report, a diorama, a sculpture). What the performance assessment asks students to do is called the task. This book is mostly concerned with writing performance tasks, not scoring schemes, although a performance assessment is not complete without both. In discussing performance tasks, I necessarily talk about what to look for in the work and how the task should connect with a scoring scheme. More comprehensive information about rubrics and other scoring schemes appears in How to Create and Use Rubrics for Formative Assessment and Grading (Brookhart, 2013b).

WHEN Should I Assess Higher-Order Thinking?

The short answer to this question is "Always." In this book, I take the point of view that you should (almost) always assess higher-order thinking. Of course, you begin planning for instruction and assessment with your state standards and your curriculum's instructional goals and objectives. Most of the time, these will include some expectations for higher-order thinking. In the few instances when they don't (for example, if the goal is memorizing a multiplication table), I advocate adding opportunities for students to make connections between what you are teaching and other things they know or with their own experience.

WHERE in My Instruction Should I Plan to Assess Higher-Order Thinking?

Assess higher-order thinking during all parts of instruction and assessment, both formative and summative. You can use higher-order thinking questions in many instances—oral class discussion, quizzes, exit tickets and other classroom strategies, and tests. You can use higher-order thinking tasks in many instances as well—classroom learning activities, performance assessments, and short- and long-term projects. The most important point here is that higher-order thinking questions and tasks should be infused throughout instruction and assessment. Don't wait until students have memorized some facts and then ask them to reason with the facts as a second step. Thinking should begin from the minute you share your learning target with students.

WHY Should I Assess Higher-Order Thinking?

There are lots of reasons to assess higher-order thinking in your classroom. Here are four of them:

• What you assess is a signal to students about what you think is important to learn.
• What you assess helps define what students will, in fact, learn.
• Assessing higher-order thinking skills leads to improved student learning and motivation.
• The Common Core State Standards and other next-generation standards require teaching and assessing higher-order thinking.

I'd like to say a little more about those last two reasons.

Thinking and Learning

Research suggests that students who are asked to think learn better. Higgins, Hall, Baumfield, and Moseley (2005) reviewed research studies that looked at the effects of thinking-skills interventions on student cognition, achievement, and attitudes. They used the method of meta-analysis, reporting effect sizes (the amount of change in standard deviation units) for the studies. For the purposes of their review, Higgins and colleagues defined thinking-skills interventions as "approaches or programmes which identify for learners translatable mental processes and/or which require learners to plan, describe, and evaluate their thinking and learning" (p. 7).

The researchers found 29 studies, mostly from the United States and the United Kingdom, that were appropriate for their investigation. These studies were conducted in primary (9) and secondary (20) schools, and most were in the curriculum areas of literacy (7), mathematics (9), and science (9). The researchers found that the average effect sizes of thinking-skills instruction were as follows:

• 0.62 on cognitive outcomes (for example, verbal and nonverbal reasoning tests), over 29 studies.
• 0.62 on achievement of curricular outcomes (for example, reading, math, or science tests), over 19 studies.
• 1.44 on affective outcomes (attitudes and motivation), over 6 studies.

Because of the small number of effect sizes of affective outcomes, the average effect-size estimate of 1.44 may be less reliable than the other two effect sizes, which are drawn from a larger number of studies. And although an effect size of 0.62 may seem small in comparison to 1.44, even 0.62 is a large effect for an educational intervention.

Abrami and his colleagues (2008) did a meta-analysis of thinking-skills interventions on measures of critical thinking itself. Although critical thinking is not exactly the same thing as higher-order thinking as defined here, there is much overlap; critical-thinking skills include "interpretation, analysis, evaluation, inference, explanation, and self-regulation" (p. 1103). Abrami and his colleagues found 117 studies with participants ranging from elementary school students through adults. The average effect size of thinking-skills interventions on developing thinking skills was 0.34 over all the effects in all the studies. However, average effects on the thinking skills of elementary students (6- through 10-year-olds, 0.52) and secondary students (11- through 15-year-olds, 0.69) were higher than were those for undergraduate college students (0.25).

Taken together, these two meta-analyses suggest that teaching thinking skills to elementary and secondary students affects the development of those same thinking skills, achievement in school subjects, and motivation, all at about the same level. The effect size of 0.62 that Higgins and his colleagues found for achievement in school subjects is equivalent to moving an "average" class of students from the 50th percentile to the 73rd percentile on a standardized measure, such as a reading or math test.

Standards

The Common Core and other standards require higher-order thinking. The standards-based reform movement actually began in the 1980s as a reaction to the "minimum competency" movement in the 1970s that had emphasized basic skills. Educators and the public quickly discovered that "the minimum became the maximum," and that minimum-competency testing on basic skills was actually de-skilling students (Brookhart, 2013c). In the 1980s, states and professional organizations began working on standards that included higher-order thinking. Parallel developments occurred in performance assessment, requiring students to use knowledge instead of simply recalling it.

At this point, because of the Common Core State Standards (www.corestandards.org), the Next Generation Science Standards (www.nextgenscience.org), and the Framework for Social Studies State Standards (www.socialstudies.org/c3), it is possible to describe some of the specific thinking skills that are or will be required for most students in the United States. But a cursory treatment here will not do the standards justice. Make sure to investigate standards in the areas you teach, probing in much greater detail than what I include here. Nevertheless, the lists that follow do show that the thinking skills required in current standards for student achievement match the two big ideas in the definition of higher-order thinking—namely, that higher-order thinking happens when students engage with what they know in such a way as to transform it, and that real knowledge is complex, effortful, generative, evidence-seeking, and reflective.

The Common Core State Standards for English Language Arts & Literacy in History/Social Studies, Science, and Technical Subjects (National Governors Association Center for Best Practices [NGA Center] & Council of Chief State School Officers [CCSSO], 2010a) organizes standards as progressions across grades within anchor standards. For reading (both literature and informational text), those anchor standards are as follows:

• Key ideas and details.
• Craft and structure [of the text].
• Integration of knowledge and ideas.
• Range of reading and level of text complexity.

For writing, the anchor standards are as follows:

• Text types and purposes.
• Production and distribution of writing.
• Research to build and present knowledge.
• Range of writing.

You can see from the wording of the anchor standards that the emphasis is on purpose-driven understanding of what a student reads or writes. A key underlying concept is the relationship among the text, other "texts" (both written texts and other knowledge and experiences), and oneself (the reader or writer). All of these pass our two-pronged test of "higher-order" thinking: using knowledge to transform, and in the process, seeking evidence, reflecting, and generating new knowledge.

The Common Core State Standards for Mathematics (NGA Center & CCSSO, 2010b) lists two kinds of standards: content standards and mathematical practice standards. Although thinking is an integral part of the content standards themselves, the mathematical practice standards, like the reading standards, are an explicit and intentional list of the kinds of mathematical thinking that students should learn how to do. There are eight mathematical practice standards that cut across grade levels:

• Make sense of problems and persevere in solving them.
• Reason abstractly and quantitatively.
• Construct viable arguments and critique the reasoning of others.
• Model with mathematics.
• Use appropriate tools strategically.
• Attend to precision.
• Look for and make use of structure.
• Look for and express regularity in repeated reasoning.

Once again, the twin themes of using knowledge to transform, and in the process, seeking evidence, reflecting, and generating new knowledge, are in evidence.

The Next Generation Science Standards are based on the National Research Council (NRC)'s Framework for K–12 Science Education (2012). Like the Common Core State Standards for Mathematics, the science standards contain an explicit and intentional list of the kinds of scientific thinking that students should learn how to do. They are called "science and engineering practices," and their goal is to "cultivate students' scientific habits of mind, develop their capability to engage in scientific inquiry, and teach them how to reason in a scientific context" (p. 41). The NRC notes that the field of science has had a historical tension between a narrow focus on teaching scientific facts, which leads to a naïve and incomplete understanding of science, and teaching scientific inquiry and reasoning. Wisely, the NRC opted for both. Actually, the framework has three major foci: science and engineering practices, crosscutting concepts, and disciplinary core ideas. Although higher-order thinking is an integral part of all three, the science and engineering practices make it particularly clear that the science standards require higher-order thinking. Here are the eight practices:

• Asking questions (for science) and defining problems (for engineering).
• Developing and using models.
• Planning and carrying out investigations.
• Analyzing and interpreting ...