Developing Evaluative Judgement in Higher Education

Assessment for Knowing and Producing Quality Work

David Boud, Rola Ajjawi, Phillip Dawson, Joanna Tai, David Boud, Rola Ajjawi, Phillip Dawson, Joanna Tai

• Publisher: Routledge
• Year: 2018
• Print ISBN: 9781138089341
• eBook ISBN: 9781351612517
• Edition: 1
• Topic: Education
• Subtopic: Education General

A key skill to be mastered by graduates today is the ability to assess the quality of their own work, and the work of others. This book demonstrates how the higher education system might move away from a culture of unhelpful grades and rigid marking schemes, to focus instead on forms of feedback and assessment that develop the critical skills of its students.

Tracing the historical and sociocultural development of evaluative judgement, and bringing together evidence and practice design from a range of disciplines, this book demystifies the concept of evaluative judgement and shows how it might be integrated and encouraged in a range of pedagogical contexts. Contributors develop various understandings of this often poorly understood concept and draw on their experience to showcase a toolbox of strategies including peer learning, self-regulated learning, self-assessment and the use of technologies.

A key text for those working with students in the higher education system, Developing Evaluative Judgement in Higher Education will give readers the knowledge and confidence required to promote these much-needed skills when working with individual students and groups.

SECTION 1
Conceptualising evaluative judgement

CHAPTER 1
Conceptualising evaluative judgement for sustainable assessment in higher education

Rola Ajjawi, Joanna Tai, Phillip Dawson and David Boud

How can we usefully think about developing evaluative judgement? This chapter introduces a framework for evaluative judgement that underpins discussion in subsequent chapters. We start by identifying what we mean by evaluative judgement and examining why it is a necessary idea for organising courses and designing assessments. We suggest that the development of students’ evaluative judgement is an important and underexplored aspect of assessment which enables us to consider the longer-term influence of assessment activities on learning. We then describe the historical evolution of the term “evaluative judgement”, and its related antecedents in the assessment literature. In one sense, evaluative judgement may not be a new concept, in that many graduates would have developed this capability. However, we argue that considering it explicitly and systematically, and using it as an integrative and organising framework for designing assessment in higher education, contributes an important and novel perspective. We conclude by examining how common assessment practices – such as rubrics, exemplars, self-assessment, peer assessment and feedback – can be better tailored towards developing students’ evaluative judgement.

We have defined evaluative judgement as “the capability to make decisions about the quality of work of self and others” (Tai et al. 2017, 5). What this definition focuses on is the need for learners to understand what constitutes quality and how to make judgements about their own work and those with whom they operate. It is a vital element of being able to work and learn in any context – in study, employment, and more broadly in society. Evaluative judgement is required to judge one’s own work, and to determine if a colleague’s work is good enough.

Why evaluative judgement and why now?

The ability to engage effectively in lifelong learning is a crucial twenty-first-century capability. Graduates are facing an uncertain future, with the rapidly changing nature of work and workplaces, multiple career shifts, and evolving roles and responsibilities (Oliver 2015). The future for graduates is less clear, and more than ever the role of education is to provide capability to learn and adapt. However, the current pace of change is unprecedented and exceeds the pace at which higher education is changing. Recent reports of workforce trends, such as the Committee for Economic Development of Australia’s report (den Hollander 2015), indicated that jobs of the future are changing more rapidly and unpredictably than they have in the past, due to technological changes and disruption. Oliver (2015, 63) argues that students and graduates need to be able to “discern, acquire, adapt and continually enhance the skills, understandings and personal attributes that make them more likely to find and create meaningful paid and unpaid work that benefits themselves, the workforce, the community and the economy”. This is where evaluative judgement can play an important role.

In the increasingly massified context of higher education, assessment and feedback practices have been described as “resistant to change” (Ferrell 2012) and are frequently critiqued for being unidirectional and content-focused, and for positioning students as passive recipients (Boud and Molloy 2013; Ajjawi and Boud 2017; Nicol 2010). Traditional assessment structures in higher education privilege the primacy of teachers, as experts, making judgements about the quality of students’ work. This creates dependence on the teacher in the absence of other opportunities for students to make judgements, such as in self-assessment or peer assessment (Sadler 2010). Such learnt dependency is counterproductive for the working world into which students will graduate.

The reason for developing evaluative judgement is about more than helping students to get a job (be employable) or be job ready (employability); it is about ensuring that graduates are safe and effective in whatever roles they take on. The notion of evaluative judgement acknowledges the complexity of contextual standards and performance, and seeks to support the development of learning trajectories and mastery. The ability to produce appropriate judgements also underpins the ability to engage in feedback conversations and thus learn effectively from the multiple opportunities that present themselves in courses, workplaces and beyond. Enhancing students’ ability to accurately self-evaluate, and evaluate others, should be implemented in a way that transfers into their working lives beyond university. Within the workplace, graduates need evaluative judgement to be able to tell if their work meets standards and, more importantly, when they need to ask for assistance from others or undertake further learning.

While Sadler’s (1989) pioneering paper forecast the importance of developing students’ evaluative judgement, only with recent advances in the conceptions and practices of assessment and feedback has it been considered in more detail (Carless 2015; Nicol et al. 2014; Tai et al. 2016; Cowan 2010). Arguments have also been made about the potential for new digital pedagogies such as interactive feedback dialogue, MOOCs and flipped classroom approaches to develop related constructs such as self-determination or self-regulation (Gaševic et al. 2014; Abeysekera and Dawson 2015; Barton et al. 2016); however, there is no systematic body of evidence about effective learning designs or appropriate learning analytics that help develop evaluative judgement.

Features of evaluative judgement

Several aspects of our view of evaluative judgement should be noted. First, in order to make decisions about the quality of any work one must have an understanding of quality; that is, what enables one to distinguish good from not-so-good work. Second, necessary for understanding quality is reference to a standard (where the standard can be implicit or explicit). Standards may be written in formal documents; however, they may also exist in exemplars of good work, and understandings in the minds of expert practitioners which may be difficult to articulate. These standards of quality are contextually bound within disciplinary notions of knowledge and/or professional practice.

In order for good evaluative judgement to be developed, there should be a desire, opportunity or habit developed in learners for making these judgements, as a way of being that is contextual, social and cultural. That is, one should take into account the circumstances and expectations of the context in which work is conducted, as well as where it will be used. In making (and articulating/justifying) these judgements, students can further refine or advance their understandings of quality and their capabilities to make judgements, linking to learning beyond the task. In educational settings, learners’ evaluative judgements need to be developed so that they can apply their capability for judgement to an increasingly wider scope of work and in a greater range of contexts. Surprisingly, little work has been undertaken on how evaluative judgement comes to be developed, and how theories of learning support this concept. There is a paucity of evidence to support educational and assessment design that promotes the development of students’ evaluative judgement.

Evaluative judgement is not to be confused with self-assessment. Students may use and develop their evaluative judgement capability through engagement in tasks that involve various forms of self-assessment. However, unlike self-assessment, which is a localised activity, evaluative judgement is a capability that extends beyond the individual and any immediate piece of work. Evaluative judgement is a fundamental attribute needed by those who are competent to operate unsupervised and manage their own learning and work.

While it may be easy to agree that developing students’ evaluative judgement might be worthwhile, there are other ideas used in considering learning which touch upon what we have discussed. So, do we then need a new term? Evaluative judgement is clearly related to and sits alongside concepts such as decision-making, metacognition, self-assessment, self-regulated learning and connoisseurship. Indeed, there are overlaps, and there are chapters in this book that address each of these terms in their own right (see e.g. Chapters 6, 7, 8, 9 and 5, respectively). However, none of these ideas entirely captures the reciprocal transfer of current understandings of quality and potential for future improvement in any given work that occurs between the assessments of others and assessments of self (or vice versa), though all of these concepts may aid in understanding what “evaluative judgement capability” is, and how we can develop it. For example, self-regulation of learning includes the notion of self-assessment (monitoring and improving performance), yet it doesn’t incorporate an explicit making and articulation of judgement in relation to peers’ work. Metacognition inhabits a cognitive psychology space, oriented towards self, which doesn’t take into account disciplinary notions of being and becoming. Connoisseurship, often associated with evaluative judgement in the arts, doesn’t have the reach/appeal that it might across higher education, and does not necessarily mean that the connoisseur is a producer of work as well as an evaluator. In advancing evaluative judgement as a concept, we seek not to subsume or supersede these concepts, but to integrate elements from each that relate to decisions about quality into a core capability for all university graduates. Each of these concepts comes from a particular disciplinary origin, and we have found that they do not sit as easily in the discourse of academics from other disciplines as the notion of evaluative judgement.

The historical development of evaluative judgement

A key step in introducing evaluative judgement to discussion of learning and assessment in higher education can be found in the work of Sadler (1989), who argued for the role of formative assessment in developing students’ “evaluative knowledge” and “evaluative expertise”. These were posited as capabilities that students must progressively develop to become independent of their teachers. Sadler argued that for students to improve, they need to develop the capacity to monitor the quality of their own work. This capacity is underpinned by an appreciation of what high-quality work is (i.e., an understanding of standards, criteria and quality), which is necessary for students to appreciate feedback about their performance. Students then need to develop “the evaluative skill necessary for them to compare with some objectivity the quality of what they are producing in relation to the higher standard, and that they develop a store of tactics or moves which can be drawn upon to modify their own work” (p.119). Academics could promote this through engaging students in “direct” and “authentic” evaluative experiences, in order to develop a concept of quality and the “facility” to make complex, multi-criterion judgements (Sadler 1989). For assessment to be transformative, students need to know what high-quality work looks like and how their work compares with that standard, and have the ability to improve their work.

The desire to move students beyond the protected environment of courses and teaching (a necessary condition of work following graduation) is encapsulated in Boud’s (2000, 152) concept of sustainable assessment: “assessment that meets the needs of the present without compromising the ability of students to meet their own future learning needs”. The focus of sustainable assessment is on helping students become more effective learners, through engagement in carefully designed assessment activities that meet the current requirements and build their capacity to judge their work beyond the immediate assessment context. This work was progressed through consideration of how assessment might be designed for future learning. Boud (2007) proposed five steps for promoting what he termed students’ informed judgement (closely akin to evaluative judgement) from the perspective of the learner, which consisted of: (1) identifying oneself as an active learner; (2) identifying one’s level of knowledge and the gaps in this; (3) practising testing and judging; (4) developing these skills over time; and (5) embodying reflexivity and commitment.

In his later writing, Sadler (2010) suggested that in order for students to make the relevant complex appraisals in relation to an assessment task, they need to grasp and apply three related concepts: the assessment genre, quality and criteria. He argued that the development of these capabilities should be one of the purposes of assessment in higher education. Sadler’s notions of developing students’ appraisal were picked up by Nicol (2010), who critiqued the predominantly monologic way feedback is delivered in the higher education sector and instead proposed that feedback should serve the function of progressively enabling students to better monitor, evaluate and regulate their own learning, independently of the teacher. Tutors’ tacit knowledge around assessment was the result of multiple evaluative judgements and seeing different ways of addressing the task (Nicol and Macfarlane-Dick 2006). Therefore, tutors developed an internal calibration for quality and comparability through these activities of reading, critiquing and evaluating pieces of work. Similarly, students should be given the opportunity to take part in the processes of making academic judgements to help them develop “appropriate evaluative expertise themselves” (appropriating Sadler’s 1989 term), and so make more sense of, and take greater control of, their own learning (Nicol 2010). Such structured opportuni...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Table of Contents
• List of illustrations
• List of contributors
• Acknowledgements
• Introduction: what is evaluative judgement?
• Section 1: Conceptualising evaluative judgement
• Section 2: Alternative theoretical perspectives on evaluative judgement
• Section 3: Approaches to developing evaluative judgement
• Section 4: Evaluative judgement for practice and work
• Index