Learning and Assessing with Multiple-Choice Questions in College Classrooms

Jay Parkes, Dawn Zimmaro

• Publisher: Routledge
• Year: 2016
• Print ISBN: 9781138845978
• eBook ISBN: 9781317540892
• Edition: 1
• Topic: Education
• Subtopic: Education General

Multiple-choice questions (MCQs) are a ubiquitous tool used in college classrooms, yet most instructors admit that they are not prepared to maximize the question's benefits. Learning and Assessing with Multiple-Choice Questions in College Classrooms is a comprehensive resource designed to enable instructors and their students to enhance student learning through the use of MCQs. Including chapters on writing questions, assessment, leveraging technology, and much more, this book will help instructors increase the benefits of a question type that is incredibly useful as both a learning and assessment tool in an education system seeking ways to improve student outcomes.

.

1 Introduction

So you’re using multiple-choice questions. That’s wonderful! Multiple-choice questions are one of the most researched and widespread educational measurement tools in existence. They have many, many advantages, as we’ll discuss. Maybe you already know that. Lots of faculty members choose multiple-choice for their classes because they permit broad content coverage in a short period of time; they’re easy and quick to score, which facilitates feedback to students; they can be administered easily through technological means. Actually the list of advantages is pretty long, and we’ll discuss most of those items in depth throughout this book.

We want to acknowledge, though, that maybe you’ve selected multiple-choice questions but you’re not happy or enthusiastic about it. You sound resigned. You sound like this is a corner you’ve been backed in to. You’d rather be using essay tests, or projects, or presentations, or papers or electronic portfolios to assess your students, but your classes are just too big. Maybe the licensing or credentialing exam in your field uses multiple-choice so you feel you need to also so your students are ready for that exam. Perhaps because of accreditation or institutional accountability pressures, your students need to participate in course-wide or program-wide tests which are not of your own choosing.

You’ve never really been enamored of multiple-choice questions (MCQs). Students need only recognize and not recall or construct the right answer. They can guess the right answer. Perhaps you think multiple-choice can only address lower-level thinking. And now you’ve got security issues with copies of previous semesters’ tests out there. And now it’s easier for your students to cheat on your exams.

You sound just like some colleagues of ours, or at least like these composite avatars of colleagues of ours. Do any of these faculty members’ situations sound familiar to you?

Olivia Peña has just been tenured and promoted to Associate Professor of History at a large suburban community college in the Western US, and her department chair has asked her to teach the large section of the general education required US History to 1877 with about 300 students in it.

The others who have taught the course have used multiple-choice tests, of course, but one of the reasons the chair would like Olivia to move into the course is that some changes are afoot, and he’d like fresh (and frankly, younger) eyes to have a look at it. Two major changes have occurred. First, the college is coming up on its Accrediting Commission for Community and Junior Colleges reaccreditation visit in a couple of years and the Dean has been preaching outcomes assessment really hard. Second, the Chair has noticed an uptick in cheating allegations coming out of that course. So things need to change and Dr. Peña is just the one to get them done!

Professor Preet Shah, a psychology professor, teaches at a mid-sized, four-year university located in the Midwest. Her university has a well-known distance education program that attracts learners from across the world. Her department chair has asked her to develop a fully online introductory statistics course that is applicable for both psychology majors and non-majors and will be offered for college credit. Dr. Shah previously taught this course as a face-to-face course with an enrollment of approximately 75 students.

After consulting with the distance education office at her institution, she learns that the learners who enroll in her course will include students at her university seeking an online alternative to the face-to-face version, college students working toward a degree at another institution, individuals seeking professional enrichment, and high school students pursuing dual credit. The distance education office informs her that she can expect between 800 and 1,000 students enrolled in the class for a given semester.

Professor Shah is a bit overwhelmed at the thought of teaching almost 1,000 students with such varying backgrounds and reasons for enrolling in her class. She previously has used MCQs for her in-class quizzes and has some questions on her classroom exams. She realizes that given the fully online delivery format and the number of students this course will now serve, she will need to expand her use of MCQs in her assessments.

It’s only been a couple of years since Sam Oliver was promoted to full Professor of Biology at a mid-size East Coast public doctoral level university. With that milestone behind him, he relaxed a bit, took a look around, and rediscovered his deep joy in teaching. One of his favorites is a junior-level course for majors, one of the first where they truly begin to specialize. This transition comes with an increased reading load for the students as well as higher expectations for reading critically and synthesizing those readings. As part of his recommitment to teaching, he took a workshop about writing across the curriculum, and he really liked the idea of a reading journal. With the increased expectations of students regarding reading, this seemed like a great way not only to encourage them to read the assignments before class but also to drive some of the critical reading and thinking. He’s actually found he enjoyed reading and responding to the journals, which surprised him a bit.

He’s been teaching one of several sections with about 20 students although the recent recession and attendant budget cut-backs at the university have driven the consolidation of those sections. So now he’ll be teaching about 50 students in a section. With that many students and without the support of any teaching assistants, he simply will not be able to retain the reading journals. He’s done some talking around campus to some of his other colleagues in the natural sciences and to the director of the university’s Center for Excellence in Teaching. He’s always been reluctant to use multiple-choice questions for lots of reasons but perhaps some weekly quizzes could meet some of the goals that the reading journals were meeting.

You might notice some similarities. None of these three initially chose MCQs but circumstances have led them to that choice. All of them actually have some pretty valid and compelling reasons to use MCQs. Perhaps you are in a similar situation.

Take heart! All is not lost! One of the central premises of this book is that “Assessments don’t hurt students; people with assessments hurt students.” You and Olivia and Preet and Sam don’t have to be one of those people. You’re looking glum about having decided to use MCQs because you think the die is cast. Far from it! The decision to use MCQs was the first of many you and Sam will make (or that will be made for you through default and neglect) that will determine what exactly your exams will measure and whether they benefit your students or do them some harm. You and Sam have not yet completely determined that you’ll never see critical thinking on exams. Whether your students are robbed of significant learning experiences because you have chosen to use multiple-choice questions yet depends on what you and they do before, during, and after they take the exam. You and Preet can conclude that teaching completely online to a large number of students resigns you to giving up on using assessments as learning tools or that you might be able to leverage technology in new ways to support how your students learn from their assessments. You and Olivia yet have the choice before you to administer a sterile measurement probe of learning objectives from which you and they will learn little and will be baffled by, or to engage in a layered set of learning experiences with your students as you and they together prepare for, take, and learn afterward from a multiple-choice test. This book will explore many different strategies to do this.

We don’t really mean to deepen your gloom, but were those papers and essay tests really that great? Take a moment to reflect: Have you gotten deep, engaged critical thinking on those essay tests, papers, or projects? Have you ever sat slumped over a set of papers and sighed, “They didn’t get it!”? How often have their essay responses been regurgitated textbook language, sometimes to the point of plagiarism? Olivia has seen a lot of that. Sam is very familiar with that frustration and nausea when he realized the trash can at the end of the hallway outside his office contained half of his class’s papers full of his copious, insightful comments about how to improve their work? Did those students get any more use out of those comments than they would have gotten from a “72 percent” that they read off a sheet taped to his office door? It’s what you and your students do before the assessment (regardless of format), what happens during that assessment, and what you and your students do afterwards, that steer the benefits of assessments for students. Preet’s experiences in her face-to-face course with MCQs leads her to believe that the assessment format itself does not determine the kind of thinking students do on assessments; the format is, at best, an invitation to certain kinds of thinking. That’s another central tenet of this book.

Be honest; you’ve not been totally unhappy about this choice. Part of you is looking forward to the relative ease and speed of using multiple-choice questions. Preet is intrigued by all of the automated assessment and gradebook functions that her university’s course management system provides. No more waiting 48 hours for bubble sheets to be scanned! Olivia thinks taking questions out of a textbook item bank and letting a computer do the scoring of the bubble sheets has got to be easier than grading a stack of papers. She’s right – sort of. Good assessment is hard work for you and for the students regardless of what the assessment format is. Your choice of assessment format (e.g. MCQs versus essay test) is, at best, a choice of where in the process you will spend your energy. In an essay test, Preet typically spends more time scoring responses than she does writing questions, while in a multiple-choice test, she typically spends more time writing questions than she does scoring them. But significant planning, thought, and work goes in to either. You haven’t truly made a choice about the time and energy you’ll spend assessing your students by choosing MCQs.

So take heart! You and Sam and Preet and Olivia have many, many choices yet to make which will determine how well MCQs work for you and your students. And the core tenet of this book is: MCQs and how you and your students use them can work extraordinarily well for you both.

Why Students Hate Tests, But They Won’t Hate Yours

While we’re being honest, there are endless reasons why students hate tests. We would wager, though, that the truth is that students hate poor tests. We will draw on bodies of research and vast stores of experience when differentiating good, or better, tests from bad, or poor ones. And, given our maxim that “Assessments don’t hurt students; people with assessments hurt students,” students don’t much like uninformed or perhaps even unscrupulous test-givers. There are unscrupulous test-givers out there; instructors who play tricks on students and may even be dishonest. We will address unscrupulous test-giving. Much, much more common is what, for ease of reference, we’ll refer to as the “uninformed test-giver”. Teaching and assessing students draw on your own values and philosophies as a person and as an educator, and yet, among the professorate, test-givers with explicit values and philosophies which they can articulate for themselves, their students, and anyone else who asks are rare. In the absence of such explicit, articulate, systematic principles, we tend to make ad hoc decisions about assessment, ones which change from student to student and occasion to occasion – even question to question on a test. We are well-meaning, perhaps even altruistic, and yet we will be less effective test-givers if we are uninformed, un-thoughtful, un-systematic and without explicit principles. (Beneficence – giving students the benefit of the doubt – is not always the best answer.)

Congratulations to you! By picking up this book, you’ve shown yourself to be at least well-meaning and perhaps even well on your way to forming or honing your own explicit, systematic values, philosophies, and principles which will guide your test-giving.

Let’s have a look at what makes tests poor and what uninformed test-givers do. To be honest, there are endless ways that tests can be poor and test-givers can be uninformed. Let’s focus on a couple of central reasons.

“(Why) Do I Have to Take This Test?”

First, tests tend to be poor tests when no one’s quite sure why the test is being taken. Why do you give your students tests? The most fundamental reason to give students a test is to ascertain whether they’ve learned what you wanted them to learn. That sounds obvious, but it is not. That sounds like a given, but it is not. That purpose is a key decision rule for you regardless of the type of assessment you are using. Think of it on a t-shirt: “Purpose is Paramount”; “The Primacy of Purpose”; “Purpose is Pre-eminent”. As you are designing an assessment, as you are asking your students to study for or perform an assessment, and as you are grading the products, every time you need to make a decision, you should ask yourself, “How will this aid my ability to know what students know?” Perhaps even more revealing would be to ask yourself this corollary: “How will this aid my students’ ability to show what they know?” For example, will the test be open book or not? Students take longer for open book tests because they look everything up. That will reduce the number of questions you can ask them. It also, in general, tests their ability to use resources and not their own reasoning or recall. Closed book exams really do drive recall and memory … and problem-solving. So which aids your ability to know what students know and for them to show what they know?

There are other good reasons to give tests. Psychologists have known for a long time that testing drives studying, that more frequent testing often results in more learning, and that how you choose to assess students impacts how they choose to study. So using tests to drive studying and learning is also an appropriate purpose for testing. We hasten to add that assessment should be one tool, but not your only tool, for encouraging and enabling students’ engagement in their own learning. The student engagement literature also suggests that assessment shouldn’t be your first tool, but that’s all beyond the scope of this book.

We would argue with you vociferously that at any college or university, at any level, in any class or credit-earning experience, those are always the top two reasons (and in that order). There are additional legitimate reasons to assess students, but they are all subsequent to, and subordinate to, those two. Assessment for selecting students into or out of subsequent courses, course development, program development, institutional accountability and reporting, accreditation needs, etc., are also legitimate, but none of those should ever overtake student learning as the primary goal of assessment inside of a course. Another way to say this is that any of those other purposes can still be served through a test which has student learning as its primary goal, and yet, if those other goals come first in your assessment thinking, it makes it harder to serve the student learning purpose.

Olivia’s college is ramping up student outcomes assessment in preparation for reaccreditation. The college is going to come to her and ask for evidence that her students are meeting the learning objectives for her course and perhaps broader learning objectives tied to the Associate of Arts degree in history and the learning objectives for the college’s core curriculum for all students. To oversimplify, she has two choices. She can work from the outside into her course by looking at those broad, core curriculum learning objectives and trying to add some assessment that neither she nor her students quite understand so that she has something to hand the college’s assessment coordinator. She can work from the inside of her course out by articulating how those broader learning objectives are carried out specifically through her own learning objectives for the course and how she assesses them for her students and herself. The former approach is going to feel forced, disconnected, and will actually decrease Olivia’s and her students’ buy in to the broader learning objectives. The latter approach meets her students’ learning needs first – including the broader objectives, which are now course-relevant – and still will provide the department and the college with the data they need to demonstrate that students are meeting the learning objectives.

Uninformed test-givers don’t really know why they give tests, or they don’t own the reasons as their own. Some test-givers give tests because, when they were a student, they took tests and so their students should, too. It’s the “test unto others as others have tested unto you” rule. That’s not very motivating for you, and it’s definitely not motivating for your students. Some test-givers would answer a student query with the Nuremberg defense: “My department chair told me I need to give this test.” While that may actually be true – you may be required to use certain assessments and/or assessment formats for reasons that are not your own – that mindset could hamper your own thinking and creativity about how you and your students can get the most out of those assessments. Voicing that reason in front of the class in response to a student question undermines your authority with your students and demotivates them. If you’re not invested in the test, why should they be?

Preet is well along in fostering this understanding. Teaching statistics, especially to people who don’t see it as directly part of their ultimate career goals, requires constant attention to “Why is this important?” So she is well-versed in articulating to students why showing your work or needing to memorize a formula, or being able to interpret output from statistical software is critical. She also has lots of experience justifying practice problems, ...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Table of Contents
• List of Figures
• List of Tables
• Preface
• 1 Introduction
• SECTION I Writing Multiple-Choice Questions
• SECTION II Assessing with Multiple-Choice Questions
• SECTION III MCQs and Leveraging Technology
• SECTION IV Teaching and Learning through Multiple-Choice Questions
• Index