Assessment, Evaluation, and Accountability in Adult Education

Lilian H. Hill, Lilian H. Hill

• Editore: Stylus Publishing
• Anno: 2020
• ISBN:
  9781620368534
• Argomento: Education
• Categoria: Evaluation & Assessment in Education

This book is intended to help practitioners in adult education become better informed about assessment, evaluation, and accountability as these are critical functions of administering and running adult education programs. The book is for adult educators who have been asked to serve on assessment committees, produce detailed reports for funders and accreditors, create a culture of assessment within their program and organization, and/or develop reports for accountability purposes.

Section one presents an introductory overview of assessment and evaluation in adult education. Section two gives guidance on practices for specific areas of adult education practice, such as military education, human resource development, and continuing professional education. Section three provides assessment practices for adults in higher education, with chapters dedicated to distance learning, health professions education, andgraduate education.

PART ONE

INTRODUCTION

1

WHAT AND WHY OF ASSESSMENT, EVALUATION, AND ACCOUNTABILITY IN ADULT EDUCATION

Lilian H. Hill

People make decisions all the time but often do not pay attention to the mental processes used to make them. We stand in the grocery aisle pondering whether to purchase one brand of peanut butter or another, quinoa or farro, kale or Swiss chard, whole wheat or multigrain bread, or to ditch the healthy stuff and go for chocolate. We may select a product based on habit, nutrition labels, or our newest diet restrictions. In our precious free time, we decide whether we would enjoy going out to a movie or staying home to garden or read a novel. Usually, we make this kind of selection quickly without much deliberation. Some decisions are of greater significance, such as whether to enroll in an education program, accept a job offer or seek another opportunity, or move to another city. When we make these choices, we are more conscious about the criteria used for decision-making.

Despite the ubiquity of making judgments in daily life, some adult educators are reluctant to approach learning about assessment, evaluation, and accountability. In fact, the topic has been neglected in the United States, whereas considerable attention is being paid to these issues in Europe, Australia, and New Zealand. Although other fields have assessment texts, no books have been published in the past decade in the United States that focus on student assessment or program evaluation in adult education. The need for a text that addresses assessment, evaluation, and accountability in adult education is urgent. Stakeholders in adult education are demanding accountability through the use of direct measures, meaning those that involve asking students to demonstrate what they have learned through course assignments (e.g., homework assignments, examinations and quizzes, research papers, presentations, role playing, artistic expression) designed to measure students’ achievement of course objectives rather than self-reported or indirect measures, such as student course evaluations, time activity logs, and unsolicited feedback in the form of cards or e-mails. Dwindling funding for some forms of adult education, especially literacy education, coupled with current accountability standards in which programs must show their worth in quantifiable and financial terms, such as return on investment, mean that adult educators must develop their skills to advocate for valuable programs with language that is most likely to be heard. Decreases in funding compared to the need is particularly visible in areas that attract government funding such as literacy education, General Educational Development (GED™) preparation, and remedial education for adults in community colleges. However, lack of funding and demands for accountability are also pervasive in other forms of adult education in which programs must demonstrate their worth in quantifiable metrics.

Existing texts on assessment, evaluation, and accountability that are specific to adult education are dated. For example, three texts that address assessment of adult learning were published in the late 1990s (Moran, 1997; Rose & Leahy, 1997; Vella, Berardinelli, & Burrow, 1998). Books that address assessment and evaluation in specific areas of adult education practice include adult literacy (U.S. Office of Educational Research and Improvement, 2002), continuing education (Knox, 2002; Queeney, 1995), informal learning (Mejiuni, Cranton, & Táiwó, 2015), and adult basic education (Bingman, Ebert, & Bell, 2000; Campbell, 2007; Grotelueschen, Gooler, & Knox, 1976; Merrifield, 1998). Although books have been published that address assessment, evaluation, or accountability in other disciplines, including higher education, student affairs, online learning, counseling, psychology, and special education, there are few accessible texts that address student and program assessment in adult education. No existing books tackle assessment, evaluation, or accountability from a critical perspective that examines how control of adult education is asserted through government policy and agency funding.

Many educators in adult and higher education have had to learn assessment, evaluation, and accountability skills in the course of conducting their work. These skills include fostering student learning, employing data to document student and program success, using validated scales for diagnostic purposes, employing formative and summative assessments to evaluate student and program success, discerning when assessment procedures are constructive, and using assessment and evaluation data for program improvement. We may need to defend a program’s continued existence or write reports for regulatory or funding agencies and other stakeholders. Not all graduate programs in adult education directly address these learning needs, and there are few resources available specific to adult education’s diversity that support learning how to conduct effective student and program assessment and evaluation.

Purpose

Assessment and evaluation are critical educational responsibilities. They focus on producing information to guide changes that foster student learning (Kuh et al., 2015). This activity has been in place for K–12 education for many years, which has in turn influenced higher education practices, and assessment and accountability demands are now permeating all forms of education including adult education (Banta & Palomba, 2015; Cumming & Miller, 2017; Hill, 2017; Kuh et al., 2015). This book is meant to be a resource for people working in adult education with responsibilities for assessment and evaluation, particularly those who are new to such responsibilities. It also provides guidance for more experienced practitioners to remain current with trends influencing the need for and format of assessment, evaluation, and accountability processes. It could be used as a textbook in adult education courses. Adult educators who have been asked to serve on assessment committees, produce detailed reports for funding and accreditors, create a culture of assessment in their program and organization, and develop reports for accountability purposes should find it helpful. Overall, this book is intended to help adult educators become better informed about assessment, evaluation, and accountability as these have become critical functions of teaching adults as well as administering adult education programs.

Definitions

Assessment and evaluation can be employed to contribute to student learning and foster program viability. The terms assessment and evaluation tend to be used interchangeably. For the purposes of this book, assessment refers to the measurement of individual student learning that may be used for screening, diagnosis, providing feedback, monitoring progress, and designing educational interventions. Evaluation involves the application of learning assessments to make judgements for program improvement and providing information to stakeholders, regional and professional accrediting bodies, and accountability systems (Galbraith & Jones, 2010). Accountability systems are designed to produce evidence that education was conducted appropriately; progress was made; and resources, particularly taxpayer monies, were used efficiently (Tusting, 2012).

Assessment in Adult Education

Although higher education has provided essential leadership in developing many assessment practices (e.g., Banta & Palomba, 2015; Barkley & Major, 2016; Cumming & Miller, 2017; Kuh et al., 2015; Massa & Kasimatis, 2017; Suskie, 2018), these practices are more aligned with the educational participation and characteristics of younger learners. Adult students participate in learning opportunities in a more fluid way than traditional students and move back and forth among home, work, and adult education opportunities. Many of our busy, overcommitted students are involved in all three at once. Adults require learning environments that value their experiences, foster active engagement in learning, and promote meaning-making (Addae, 2016). Addae advocates for teaching to be a dialogical process in which teachers and learners alike are involved in decision-making about activities that will help learners achieve course learning objectives and construct their own meaning.

Assessment has instructional and diagnostic purposes. It enables adult educators to discern learners’ understanding of the subject matter and adjust their instruction to the learners’ needs. Addae (2016) states that adults’ interest in immediacy of application renders diagnostic assessment (i.e., identifying where students are having difficulty) critical because, through regular assessment, the instructor is able to “gather enough information about learners’ difficulties with the content of the course and as such be in a better position to address them” (p. 190). Addae promotes several principles for planning assessments: (a) educators are responsible for designing assessments, but learners should be involved in their planning; (b) assessment must take place alongside instruction so the instructor can make immediate adjustments to instruction when needed; (c) questioning to promote students’ ability to reflect on learning should be an essential part of teaching; (d) instructors should be attentive to students’ ability to articulate their learning; and (e) formal assessments should entail application projects in which learners work together to solve a problem. Assessment allows learners to reflect on their learning and make their own adjustments in their efforts to learn.

Based in key principles of adult learning, Table 1.1 shows Kasworm and Marienau’s (1997) five principles to guide adult-oriented assessment of learning that remain pertinent.

TABLE 1.1
Assessment Strategies to Guide Adult-Oriented Assessment of Learning

Key Premises of Adult Learning	Key Principles of Adult-Oriented Assessment Practice
Learning is derived from multiple sources.	Recognizes multiple sources of knowing; that is, learning that occurs from interaction with a wide variety of informal and formal knowledge sources
Learning engages the whole person and contributes to that person’s development.	Recognizes and reinforces the cognitive, conative, and affective domains of learning
Learning and the capacity for self-direction are promoted by feedback.	Focuses on adults’ active involvement in learning and assessment processes, including active engagement in self-assessment
Learning occurs in context; its significance relates in part to its impact on those contexts.	Embraces adult learners’ involvement in and impact on the broader world of work, family, and community
Learning from experiences is a unique meaning-making event that creates diversity among adult learners.	Accommodates adult learners’ increasing differentiation from one another given varied life experiences and education

Note. From Kasworm and Marienau (1997). Reprinted with permission.

Assessment methods compatible with Addae’s (2016) and Kasworm and Marienau’s (1997) principles include dynamic assessment and sustainable assessment. Dynamic assessment is a system of theories that consider learning and cognition to be modifiable during interactions with other people in a sociocultural context (Robinson-Zañartu, 2013). It examines the difference between an individual’s current performance level in comparison with their capabilities (Malmeer & Zoghi, 2014). Because performance assessment is congruent with the goals and contextual experiences of adult learners, learners are able to demonstrate their learning by responding to a realistic scenario or task (National Research Council, 2002). By providing immediate feedback, this interactionist and holistic form of assessment is meant to be developmental with the purpose of improving student performance. Instead of static measurement of current ability, dynamic assessment provides learning support tailored to students’ learning needs during ongoing social interactions between learners and instructors.

Sustainable assessment (Nguyen & Walker, 2016) was designed to provide assessment for learning and relies on formative assessment with an emphasis on developing skills for future learning. The provision of sustainable feedback involves teachers in providing detailed and effective feedback; students’ active participation in the feedback process; engagement of students in dialogue with teachers about the feedback; and fostering students’ lifelong learning abilities for “self-regulation, self-evaluation, and the use of feedback from multiple sources to improve their learning over time” (Nguyen & Walker, 2016, p. 98). Boud and Soler’s (2016) framework emphasized encouraging learners to become reflexive and more accurate judges of their learning and promoting students’ ability to take a more active role in the feedback process. This approach aligns learning assessment with lifelong learning.

Assessments should move beyond measuring only the cognitive aspects of knowledge acquisition (knowing something) to include examining individuals’ growth in competence (knowing how) in the face of continuous change (Su, 2015). Assessment of competence, the “capacity to perform or act in particular contexts, . . . highlights the learner’s problem solving and completion of tasks in context, . . . required to ensure effective adaptation to contextual changes in life” (p. 78). Su further notes that assessment of learners’ competence is insufficient without assessment of learners’ commitment to continue learning, meaning “the learner’s dedication to learning and development based on his or her feeling of meaningfulness when facing the changing future to address life’s challenges” (p. 80). Assessment of competence is pragmatic, whereas assessment of commitment is existential.

Program Evaluation in Adult Education

Program evaluation entails systematically collecting and analyzing data to make informed judgments about whether a program met its objectives, how to improve its effectiveness, and how to plan for future programming (Cervero & Wilson, 2006; Fitzpatrick, Christie, & Mark, 2009). Evaluation processes may describe whether the program (a) fulfills compelling personal and societal needs, (b) merits changes in its delivery and management, (c) is responsive to the sociopolitical context, (d) is justified in its costs and use of resources, and (e) prepares students to apply their learning to their personal and work needs (Caffarella & Daffron, 2013). Depending on the purposes of evaluation, multiple methods may be used, including observations, interviews, surveys, pre- and post-tests, portfolios, focus groups, cost-benefit analyses, self-assessments, and even storytelling. Multiple methods may be appropriate depending on the evaluation goals. Each of these qualitative and quantitative methods have strengths and weaknesses, so evaluators must make careful decisions about the types of evaluation data they will collect and analyze.

When planning programs, some people fail to plan for program evaluation in advance and resort to devising a so-called happy sheet that measures participants’ satisfaction with the facilities, refreshments, handouts, content, and delivery ...