The Student Assessment Handbook

New Directions in Traditional and Online Assessment

Lee Dunn, Chris Morgan, Meg O'Reilly, Sharon Parry

• Publisher: Routledge
• Year: 2003
• Print ISBN:
  9781138147355
• eBook ISBN:
  9781134310135
• Topic: Education
• Subtopic: Education General

A guide to current practice in assessment, particularly for those professionals coming to terms with new pressures on their traditional teaching practices. Increased use of IT, flexible assessment methods and quality assurance all affect assessment, and the need to diversify and adapt traditional assessment practices to suit new modes of learning is clearer than ever.
The Student Assessment Handbook looks at the effectiveness of traditional methods in the present day and provides guidelines on how these methods may be developed to suit today's teaching environments. It is a practical resource with case studies, reflection boxes and diagnostic tools to help the reader apply the principles to everyday teaching.

The book provides advice on a wide range of topics including:
* assessing to promote particular kinds of learning outcomes
* using meaningful assessment techniques to assess large groups
* the implications of flexible learning on timing and pacing of assessment
* the pros and cons of online assessment
* tackling Web plagiarism and the authentication of student work
* mentoring assessment standards
* assessing generic skills and quality assurance.

Part B

Assessing key learning outcomes

Introduction to Part B

In Part A, we looked at some of the broad foundation principles of assessment and explored some contemporary issues and themes confronting teachers. Having established this framework, we now focus on an area of key concern in assessment – choosing the right methods to assess students’ achievements.

There is no shortage of books on assessment methods and tasks. What’s different about this book? Our concern with the traditional focus on methods is that it tends to overlook the vital importance of the disciplinary context and the overall learning outcomes that teachers want to achieve. Using a traditional ‘A to Z of assessment methods’ text, newcomers to higher education are likely to shop around for methods that they ‘like’ or those that they are most familiar with from their own undergraduate days. They may tend to view assessment as an add-on event, something additional, rather than integral, to teaching and learning.

In this part, we argue that, when selecting methods for assessment, it is vital for teachers to be clear about the disciplinary and generic skills and attributes that they are developing, the overall aims of the programme, and the particular aims and desired learning outcomes in their subject. If assessment tasks are not congruent with these broader goals, if they are not tailored most explicitly to draw from students the desired outcomes, then, at best, it is an opportunity lost.At worst, it can prompt counter-productive learning and may confuse and frustrate students.

For example, an essay can be chosen as a method of assessment for a variety of reasons. It may be used to develop and assess students’ analytical and evaluative abilities, or it may be shaped primarily to test students’ written communication skills and ability to order information coherently. Similarly a journal may require students to evaluate and synthesize experience, or perhaps simply to observe and document phenomena. Group work may be facilitated to develop and assess problem-solving skills or to develop students’ ability to negotiate difference. Often it is not so much the assessment method that counts, but the particular emphasis that it is given, which prompts the appropriate responses from students.This emphasis particularly includes the rationale and marking criteria that convey to students what is really wanted in an assessment task and why.

For this reason, we have steered away from ‘methods’ as the primary focus of this part. Rather we have looked at methods through the lens of learning outcomes. In this respect, we are greatly indebted to the efforts of Nightingale, TeWiata, Toohey, Ryan, Hughes and Magin in their 1996 publication, Assessing Learning in Universities.Their project entailed collaborative research and consultation to determine the kinds of broad learning outcomes considered desirable across programmes of higher education in Australia, and devised an excellent organizing principle for assessment based on clusters of abilities as follows:

• thinking critically and making judgements;
• solving problems and developing plans;
• performing procedures and demonstrating techniques;
• managing and developing oneself;
• accessing and managing information;
• demonstrating knowledge and understanding;
• designing, creating, performing;
• communicating.

In addition to our debt to Nightingale et al, we also draw on the work of Becher (1989), whose contribution to our understanding of the nature of knowledge across disciplinary settings is infused into each of the eight learning outcomes. We acknowledge that each learning outcome will be expressed differently in the different disciplinary settings of the sciences, the humanities, the social sciences and the applied professions, and that transdisciplinary fields of study are even more complex with respect to intended learning outcomes.

We have drawn heavily on the organizing principle of Nightingale et al (1996) throughout Part B, applying our own internal structure as follows:

1. Introduction – a broad overview of the particular ability or learning outcome. What is it and why is it valued?
2. Learning outcome in the disciplines – an exploration of the different forms that the learning outcomes might take in the disciplines.The aim of this section is to reinforce the vital role of context and disciplinary convention in relation to the forms that assessment takes.
3. Assessment methods – an overview of the range of assessment methods commonly employed to assess this learning outcome, and how they are facilitated. We do not assume a face-to-face teaching context. Where relevant, we refer to distance and online variations of methods.
4. Cases – the cases provide windows into how particular methods have been applied across disciplines. They are not fully worked case studies with all details of the assessment schemes. Instead they are brief snapshots intended to illustrate a particular method with sufficient detail for teachers to adapt it to their own context.The cases are written by the authors as a synthesis of our practice and that of our colleagues.Where they represent someone else’s work in large measure, then appropriate citations have been made.
5. Implications for teaching – this is the final piece that completes the picture. It is difficult to separate assessment and teaching – they are natural allies.We take the view that assessment tasks are only purposeful to the degree that they are supported by congruent teaching. By the time students undertake a task, they should be clear about what’s expected of them and have had the opportunity to discuss and practise the task with developmental feedback.

The only significant difficulty with the organizing principles adopted here is one of overlap.The same assessment method may apply to a number of learning outcomes although, as discussed, the emphasis is often very different.As this is a handbook that is really designed for dipping in and out of as needs be, we feel it is better to have some overlap and minor repetition rather than complex cross-referencing systems.

12

Communicating

Introduction

Every assessment task is an exercise in communication.Teacher judgement is, to a large extent, based on how well students communicate their responses to a task, and, from the students’ point of view, success partly depends on whether they understand how to approach their assessment tasks. Undergraduates are expected to learn academic conventions of communication, particularly those in their disciplinary fields, in order to express knowledge and demonstrate the achievement of desired learning outcomes in any of the domains of learning.

To say that communication in one form or another is all-pervasive in higher education, as in every sphere of life, is to state the obvious. Communication conventions are continuously assessed all the way through undergraduate and postgraduate education, and grades have been shown to reflect whether there is a match of structure and style between teacher expectations and student responses to assessment tasks. A study of engineering students’ oral presentations (Dannels, 2002) found that grades depended partly on how well students used a particular results-oriented structure to organize their oral presentations. Students can stand or fall on how they express their knowledge and ideas, but learning outcomes and marking criteria may not always specify expectations about how they should use academic communication styles to improve their results.

Many of the conventions governing how students should communicate orally or in writing are not explicit, although disciplinary conventions for writing and speaking are understood and used by academics and experienced students (Parry, 1998). A study by James (2000: 354) showed that some transferable communication skills, for example the way language is used in a ‘good’ essay, or the structure of a ‘good’ oral presentation, are a matter of ‘collective received wisdom’ and might not appear in the marking criteria given to students before they attempt the task.

The characteristics and transferable features of, for example, a good essay or an effective oral presentation can be difficult to articulate (James, 2000). Such conventions are sometimes considered to be generic, but many studies have shown that disciplinarity has a huge influence on how knowledge is expressed in higher education (Neumann, 2001). Students also want to know what their particular lecturers and tutors are looking for in relation to how they should communicate in their assignments and examinations because there are individual differences of emphasis among academics as well as between disciplines.

What is communicating?

Human communication is about imparting or exchanging thoughts, opinions or information by speech or writing, in visual form or by some other means. Effective communication evokes understanding of what the originator has attempted to impart (Moore, 1993). In higher education, it is also about how knowledge is conveyed to a listener or a reader, in a variety of ways using different media (Nightingale, 1996a). Communication also occurs through works of art, performance and drama to communicate aesthetic values, ideas or emotion. Academics and students communicate their ideas in a great many ways, including:

• oral communication: tutorial or client presentation, viva voce, debate, argument, advocacy, moot court, interview, clinical interaction;
• written communication: essay, report, diary, journal, creative writing, written accompaniment to an oral presentation, presentation of scientific results;
• visual communication: posters, graphics, PowerPoint slides, overhead transparencies, technical drawings, diagrams, charts or other illustrations in hard copy or online;
• online communication: synchronous and asynchronous communication through e-mail, discussion groups, student intranet activities.

These methods of communication are complex and need to be situated in their context and the academic purpose that derives from curriculum objectives and assessment tasks. Also, the ways in which students can demonstrate communication skills vary according to the mode of educational delivery.Teachers have to design ways to assess oral communication (if learning outcomes require it) for distance and online students as well as those who can attend classes on-campus. For instance, telephone conferencing can be used for oral examinations when students cannot attend face to face.

Online communication has reduced the isolation felt by distance students, especially when small groups work as project or study groups. Distance students can be part of intellectual and learning communities in a way that was not possible when their communication was paper based (Parry and Dunn, 2000). Yeoman (1996) has argued that students will communicate online whether it is assessed or not, because the process can be so stimulating for them. Her findings suggest that not only is assessment a powerful tool for indicating to students what they should learn but, in online courses, such communication can also be used to direct how they should learn. To be effective, though, online communication and group collaboration in this medium should be task based, and this requires teacher organization and the allocation of roles to students (Pincas, 1997).

Communicating in the disciplines

Induction and socialization into a discipline is mostly conducted through its ways of communicating and using its language. Most frequently, for students, their learning is focused on writing in a way that is acceptable in the field. Language is of immense importance to disciplinary groups of academics, because their individual and collective knowledge, values and traditions are interpreted and expressed through language (Becher, 1989).

Communication in disciplinary contexts might not be taught overtly, but is learnt by students in several ways: through trial and error when appropriate style is rewarded, with feedback on how to improve, by emulating the type of communication that is valued by their academic teachers or by becoming familiar with scholarly journals in their field. Learning how to communicate in higher education is not about developing skills in isolation, but about acquiring knowledge about how and why particular communication is valued and also about the application of that knowledge to practical situations (Burton and Dimbleby, 1990).

Academic writing is mysterious to many undergraduates until they discover the overarching principles that they must follow. Their teachers often cannot articulate the rules and principles, but ‘know them when they see them’. According to Parry (1998), the three most obvious domains of writing style are:

• the way the argument is structured and styled;
• the conventions for citation and acknowledgement; and
• the type of terminology used.

In science, undergraduates spend a lot of time learning the rules of academic communication through the precise structure of laboratory reports and by seeing, from the beginning of their course, models of the way arguments should be constructed. In the other disciplinary groupings the structures are not so clear cut and can be more individualistic. In the social sciences it is a matter of finding out whether an argument tends more towards the humanities (built on theoretical or conceptual perspectives) or more towards science (built on concrete research findings) and then choosing which model to follow. How knowledge is reported changes with the nature of the knowledge base and disciplinary paradigms.The following are some examples from groupings of disciplines derived from Parry’s (1998) analysis of ways to articulate knowledge orally and in writing:

Science
In the sciences, written reports are information based and argument is developed by building new knowledge from what is previously known. Students learn to communicate within the shared theoretical paradigm of experimental science. Scientific writing is technical and concrete, and knowledge is expressed by reporting and explaining what has been found. Students learn from the beginning of their studies how to structure writing tasks in the proper way.

Social science
In the social sciences students have a more complex task in learning disc...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• About the Authors
• Foreword
• Preface
• Acknowledgements
• Part A: Issues and Themes In Assessment
• Part B: Assessing Key Learning Outcomes
• Part C: Assessment In Practice
• References