Grading Student Achievement in Higher Education

Signals and Shortcomings

Mantz Yorke

• Publisher: Routledge
• Year: 2007
• Print ISBN: 9781138975408
• eBook ISBN: 9781134161454
• Edition: 1
• Topic: Education
• Subtopic: Education General

A lot hangs on the summative grades that students are given. A good degree opens doors which otherwise might remain closed. Yet, as higher education is now a mass rather than an elite system, what is expected of its graduates is different from the expectations of previous generations. Students are expected not only to be able to demonstrate high standards of academic achievement, but also a variety of capabilities that have at different times been given labels such as 'generic skills' and 'transferable skills'. These abilities are difficult to grade for a variety of reasons and some graduates may be losing out because their particular strengths are given insufficient acknowledgement in current summative assessment practices.

Using the UK honours degree classifications as a case study, this book appraises the way in which summative assessment in higher education is approached and shows that the foundations of current practices (in the UK and elsewhere) are of questionable robustness. It argues that there is a need to widen the assessment frame if the breadth of valued student achievements is to be recognised adequately.

Chapter 1

The complexity of assessment

Introduction

Student achievement is assessed for a variety of purposes, some of which are in tension with others. This multiplicity of purposes engenders compromises that are not always helpful to the assessor. This chapter clears the ground for more detailed examination of grading in the subsequent chapters. There are plenty of books covering various aspects of assessment (e.g. Brown and Knight, 1994; Brown et al., 1997; Walvoord and Anderson, 1998; Heywood, 2000; Knight and Yorke, 2003; Walvoord, 2004) to which the reader can turn for discussions of approaches to assessment; because the emphasis of this book is on grading and its implications, this chapter is limited to a brief overview of a number of the main aspects of assessment. These include formative and summative assessment, with emphasis being given predominantly to the latter; norm- and criterion-referencing; and technical issues in assessment. The approaches that are adopted in respect of summative assessment, and their technical quality, influence the approach taken in this book towards the long-running debate on the extent to which achievements can be measured, or have to be judged.

Purposes of assessment

Students are assessed for three main reasons: to promote learning; to certify achievements; and to provide data that can be used for quality assurance (sometimes quality control) purposes (Table 1.1).

Boud (2000: 159) refers to assessment doing ‘double duty’ – the ostensible and the tacit. His elaboration and Table 1.1 suggest the multiplicity of purposes of assessment under both the ostensible and the tacit. Hounsell (2007) summarizes neatly the tensions inherent in assessment.

[Assessment] is called upon to be rigorous but not exclusive, to be authentic yet reliable, to be exacting while also being fair and equitable, to adhere to long-established standards but to reflect and adapt to contemporary needs, and at one and the same time to accommodate the expectations not only of academics, their students and the university in which both are engaged, but also of government and government bodies, . . . employers, professional and accrediting organisations, subject and disciplinary associations, parents, and the public at large.

Table 1.1 Purposes of assessment

Hounsell goes on to suggest that the most challenging tension is probably that between summative and formative assessment.

Summative and formative assessment

Summative assessments are couched in what Boud (1995) terms ‘final language’, since they sum up the achievements of students. The certification of achievement is a summative, ‘high stakes’ matter for students in that it possesses direct implications for their futures. Such certification has to be robust in that it has to demonstrate such technical qualities as high validity and reliability. Some summative assessments may not, in themselves, be particularly ‘high stakes’ in character. They may count relatively little towards an overall grade computed in respect of a whole course. A group presentation, for example, may be given a percentage mark, yet be weighted to a relatively small extent in the grade awarded for a module. Further, marks for some kinds of task cluster closely and may have a very limited influence on an overall module grade.

Formative assessment does not necessarily have to reach the level of technical quality that is expected of summative assessment, since its primary purpose is to encourage the learner, in one way or another, to develop their capacity to meet the challenges that face them. Greenwood et al., for example, say that formative assessment

implies no more (and no less) than a discerning judgement about [a] learner’s progress; it is ‘on-going’ in the sense that it goes on all the time; and it is formative in so far as its purpose is forward-looking, aiming to improve future learning (as distinct from the retrospective nature of summative assessment).

(Greenwood et al., 2001: 109)

Formative assessment is dialogic, conversational in intent, seeking to engage the student in identifying ways in which performance can be improved – and acting on the enhanced understanding. For some students, the challenge may be to repeat a task on which they have been adjudged to have failed; for others, it may be to do better on the next task that faces them. Formative assessment is, in principle, ‘low stakes’ since it is concerned with development much more than it is with grading – indeed, formative assessments may not involve any grading.

However, some assessments are both formative and summative. Assessments that take place within modules may be both formative, providing feedback on performance, and summative, in that they count towards the grade to be awarded for performance on the module as a whole. Examples of such assessments are multiple-choice tests, formal class quizzes and short assignments (which may cumulate to fulfil the assessment requirements in a manner such as the ‘Patchwork Text’ described by Winter, 2003). It is for this reason that seeing high stakes as relating to summative, and low stakes to formative, assessment is an over-simplification.

This book focuses on summative assessment, which has become increasingly challenging as the expectations placed on higher education have evolved.

Some issues in summative assessment

Coursework v. examination

It must be borne in mind that ‘coursework’ and ‘examination’ are portmanteau terms within which there can be considerable variation. Coursework can span a range from extended projects to short pieces of work such as a 500-word review, and examinations can include not only the traditional unseen examination paper, but also papers made available to students before the day of the examination, when they have to respond without resources to hand, and ‘open book’ examinations, all of which are conducted under tightly controlled conditions.

The point is frequently made that coursework tends to attract higher grades than examinations, and there is empirical data to this effect (e.g. Bridges et al. , 1999, 2002; Yorke et al., 2000; Simonite, 2003). Simonite’s study pointed up the significance of the difference when she noted that, in the case of Biology and Molecular Sciences, if four modules that counted towards the honours degree classification switched from a mixture of examinations and coursework to coursework only, this would on average raise a student’s mean mark by 0.7 of a percentage point – enough to influence a number of classifications across the student cohort.

A second effect of making more use of coursework in assessment is a tendency to narrow the range of marks, as Simonite points out. However, whereas some students might gain from a shift towards coursework, others might fare less well. She raises the question of what constitutes fairness in assessment.

The widening frame

A central issue in higher education is what summative assessment is expected to cover. In the UK of the 1960s and 1970s, summative assessment focused upon academic achievements related to the subject discipline being studied. Gradually the terms of reference of assessment widened, under governmental prompting based on human capital theory, to include concepts such as enterprise and employability. Whereas it may have been relatively easy (though not as easy as some seem to believe) to classify performances at bachelor’s level in terms of academic achievement, it becomes much more difficult to do this when the assessment requirements cover a much wider spectrum of achievements, some of which may not be amenable to reliable representation in grades.

Academic and vocational education have often been depicted as different within higher education, yet there are many examples of vocational programmes which are accepted as ‘academic’ (those in Medicine, Law and Engineering are three). In others, such as Teacher Education and Social Work, the perception is arguably of less academicism and more vocationalism.

Employability

Stephenson (1992, 1998) argued a case for ‘capability’ in higher education at a time when academics in the UK had, in the main, yet to come to terms with the government’s promotion of ‘enterprise’ through its Enterprise in Higher Education initiative. Stephenson saw capable people as possessing confidence in their ability to take effective and appropriate action; to explain what they were seeking to achieve; to live and work effectively with others; and to continue to learn from their experiences, both as individuals and in association with others, in a diverse and changing society. He made the point that capability was a necessary part of specialist expertise, and not separate from it. Capable people not only knew about their specialisms, they also possessed the confidence to apply and develop their knowledge and skills within varied and changing situations.

If Stephenson’s conception was ahead of its time, it strongly influenced thinking in the UK about employability, seen by the Enhancing Student Employability Co-ordination Team (ESECT) as

a set of achievements – skills, understandings and personal attributes – that make graduates more likely to gain employment and be successful in their chosen occupations.¹

(Yorke, 2004/06: 8)

ESECT developed an account of employability, given the acronym USEM, which linked Understanding (of subject disciplines and situations); Skilful practices in context; Efficacy beliefs and personal qualities; and Metacognition. ESECT’s approach differed from Stephenson’s presentation of capability in that it was able to draw on a range of theoretical and empirical work to give it the kind of academic foundation which academics could respect.²

Competence

‘Competence’ is a term that is used widely in discussions of vocational programmes, and rather less in respect of academic programmes. A difficulty is that the meanings ascribed to it vary: in some contexts, such as in North America and in management, it is taken to refer to a personal attribute or quality (as is the related term ‘competency’), whereas elsewhere it refers to social expectations or judgements relating to performance.³ Either way, it is a social construct which is inflected with values (Kemshall, 1993; Lum, 1999; Hager, 2004a).

Hager (2004a) points to the need to differentiate between three particular aspects of competence: performance and its outcomes; its underpinning constituents (i.e. capabilities, abilities and skills); and the development of people to be competent performers. In the UK, there are two variations on the theme of competence: the first is the subdivision of performance, through functional analysis, into the plethora of detailed competences that characterized the system of National Vocational Qualifications (NVQs) in the 1990s (see Jessup, 1991, for a detailed account); the second is a broader interpretation in which components of performance are ‘bundled together’. Some see competence as subsuming more than can be ‘measured’ through assessment processes. Worth-Butler et al. (1994) exemplify this when they describe competence in terms of the

mastery of requirements for effective functioning, in the varied circumstances of the real world, and in a range of contexts and organizations. It involves not only observable behaviour which can be measured, but also unobservable attributes including attitudes, values, judgemental ability and personal dispositions: that is, not only performance, but capability.

(Worth-Butler et al., 1994: 226–227)

Slightly more narrowly, Hager and Butler (1996) describe competence in terms of the ability of a person to respond adequately to the range of demands that constitute a role. Jessup (1991: 27) makes much the same point. These descriptions of competence point to skilfulness in practice as encapsulated in the ‘S’ of USEM (see above).

The literature bears witness to debate about the theory underlying notions of competence. Hyland, in a number of publications (e.g. Hyland, 1994), argued that competence was rooted in behaviourism, but Hager (2004a) criticizes him for not differentiating between behaviour (the manifestation of competence) and behaviourism. Hager’s argument is consistent with an earlier defence of competence-based assessment (Hager et al., 1994) which offered Jessup’s (1991) approach to competence (with its strong affinity with the behavioural objectives approach espoused by Mager, 1962, and others) the prospect of redemption by suggesting that, if Jessup’s statements of outcomes of performance were construed as performance descriptors, they would possess an abstractness that took them some distance away from the narrowness perceived in them by most critics of NVQs. Hager et al. may have pressed their interpretation farther than can be justified.

Graduates entering the labour force do so, in most instances, as young professionals who are expected to be able – at least at a beginning level – to deal with the complex and ‘messy’ problems that life tends to throw at them. This complexity is far removed from the narrow approach favoured by Jessup (1991) and others concerned to decompose performance into narrowly focused skills to be demonstrated across a defined range of situations. Although highly disaggregated competences, such as those that were introduced in NVQs, have value in developing an understanding of the dimensions of workplace performance, there is more general support for seeing ‘competence’ in much broader terms.⁴ Eraut (2004b: 804), for example, writes:

treating [required competences] as separate bundles of knowledge and skills for assessment purposes fails to recognize that complex professional actions require more than several different areas of knowledge and skills. They all have to be integrated together in larger, more complex chunks of behaviour.

Others have made the point that competence or competency frameworks derived from functional analysis are inadequate for assessment purposes, on the grounds that they miss some subtleties of performance (e.g. Owens, 1995, in respect of social work; Jones, 2001; Coll et al., 2002; and Cope et al., 2003, in respect of teaching; Jones, 1999, in respect of vocational education and training; and Lang and Woolston, 2005, in respect of policing in Australia). The importance of judgement in the assessment of complex achievements is emphasized by van der Vleuten and Schuwirth:

As we move further towards the assessment of complex competencies, we will have to rely on other, and probably more qualitative, sources of information than we have been accustomed to and we will come to rely more on professional judgement as a basis for decision making.

(van der Vleuten and Schuwirth, 2005: 313)

A further complication is that some (e.g. Hays et al. 2002; Schuwirth et al. 2002) draw a distinction between ‘competence’ and ‘performance’. The former represents a person’s achievement under test conditions, knowing that they are being challenged to demonstrate knowledge, attitudes and skills (and is often implicitly taken to be the best that they can achieve, though not everyone gives their best performance under the stress of formal testing), whereas the latter is what the person achieves on a day-to-day basis. ‘Competence’ in these terms might be seen metaphorically as a peak whereas ‘performance’ might be seen as a broad col below peak level. Although one might perform at peak on specific items of assessment (coursework and/or examination) and be assessed accordingly, on an extended task such as a ward placement or teaching practice the assessment will quite probably integrate over the whole of the engagement and hence come closer to signalling the day-to-day level of performance.

Norm-referenced and criterion-referenced assessment

The assessment of student achievement implies some frame of reference against which judgements are made. If reference is made to the achievements of other students (whether in the same cohort or multiple cohorts) then norm-referencing is to the fore. If the reference is to stated objectives or expected learning outcomes, then criterion-referencing⁵ is of key importance.

Norm-referenced assessment

Norm-referenced assessment is relativistic, in that, in typical practice in higher education, it seeks discrimination amongst students by placing their achievements in order of merit rather than by setting them against the kinds of norms that are developed for psychological and mass educational testing. An assumption sometimes made is that the observed performances are distributed approximately normally, i.e. that the frequencies of different categories of performance fit the normal distribution, and that the performances can be grouped in bands that fit a normal distribution ...

Table of contents

• Cover Page
• Title Page
• Copyright Page
• Figures
• Tables
• Acknowledgements
• Abbreviations
• Prologue
• Chapter 1 The complexity of assessment
• Chapter 2 Grading and its limitations
• Chapter 3 Variations in assessment regulations
• Chapter 4 UK honours degree classifications, 1994–95 to 2001–02
• Chapter 5 How real is grade inflation?
• Chapter 6 The cumulation of grades
• Chapter 7 Value added
• Chapter 8 Fuzziness in assessment
• Chapter 9 Judgement, rather than measurement?
• Epilogue
• References