Academic Life in the Measured University

Pleasures, Paradoxes and Politics

Tai Peseta, Simon Barrie, Jan McLean

• Publisher: Taylor & Francis
• Year: 2020
• Print ISBN: 9781138369535
• eBook ISBN: 9780429767456
• Edition: 1
• Topic: Education
• Subtopic: Education General

While a life in academia is still one bestowed with enormous privilege and opportunity, on the inside, its cracks and fragility have been on display for some time. We see evidence of this in researchers bemoaning time spent applying for grants rather than doing research; teachers frustrated at the ways student feedback data are deployed to feed judgements about them; and doctoral students realising that they have little chance of securing full-time academic work. Yet in the public policy domain, the opposite appears true: academics left to their own devices in their elite ivory towers, rarely ever do enough.

This collection addresses the fact that academic life deserves to be rigorously researched. Its emphasis on the measured university traces how academic life had ceded itself to the logics of perverse measures, and raises questions about whether the contemporary university may well have become too measured to adequately counter the political times now upon us. The contributors explore the ways in which measurement inhabits paradoxical positions in these spaces. It sketches the contours and consequences of mismeasurement, including the personal costs to academic staff. It examines our desires and fumbled efforts at institutional transformation, and it puts on display our own ethical conduct. The collection concludes with a call to chart a course for a revitalized moral economy of academic labour.

This book was originally published as a special issue of Higher Education Research & Development.

Higher degree research by numbers: beyond the critiques of neo-liberalism

Liam Grealy and Timothy Laurie

ABSTRACT
This article argues that strong theories of neo-liberalism do not provide an adequate frame for understanding the ways that measurement practices come to be embedded in the life-worlds of those working in higher education. We argue that neo-liberal metrics need to be understood from the viewpoint of their social usage, alongside other practices of qualification and quantification. In particular, this article maps the specific variables attending measurement in higher degree research programmes, as the key sites that familiarize students with measurement practices around research and teaching. With regard to the incremental reframing of doctoral study as a utilitarian pursuit, we suggest a need to better identify the singular and immeasurable features of long-term research projects, and argue for a revitalized notion of failure.In this context, this article suggests that many critiques of neo-liberalism do not sufficiently advance alternative ways to think about the purposes and limitations of higher education.

Introduction

Like the category of obscenity in art, neo-liberalism seems to obey a maxim that ensures its regular circulation within university folklore: we know it when we see it. We know something neo-liberal is happening when generous colleagues fail to have their contracts renewed because they do not meet obscure metrics for research performance derived from Key Performance Indicators (KPIs). We know neo-liberalism has taken hold when, rather than pursuing their interests or talents, students make course choices based on perceived benefits for future employment incomes to repay higher tuition fees resulting from sector-wide marketization and deregulation. We know that we belong to neo-liberal institutions when the carefully formed languages of specialist disciplines are replaced by the languages of management, such that a research project acquires value only through its outputs, its impacts, its scalability, its ‘innovations’, or its potential to produce further growth. And yet, we do not always know what a unifying concept such as ‘neo-liberalism’ adds to all of this knowing.

Many contemporary commentaries on neo-liberalism could be classified as what Eve Sedgwick, following Silvan Tomkins, calls ‘strong theories’. Strong theories claim to be ‘capable of accounting for a wide spectrum of phenomena which appear to be very remote, one from the other, and from a common source’ (Tomkins, quoted in Sedgwick, 2003, p.134). For example, Marxist political economists have been accused of being overly ‘functionalist’ in their accounts of neo-liberalism, by attributing multiple transformations to a single causal factor – the struggle between capital and labour (Flew, 2014, p. 58). Calling Marxisma ‘strong theory’ does not mean saying that it is false, only that its most common articulations involve moving from an unstable multiplicity of effects to a more stable continuity of causes. But cultural accounts of neo-liberalism can also contain similarly ‘strong’ impulses. For example, competitive reality television programmes or self-help books have been described as neo-liberal because they participate in the common logic of individual self-determination, even if competitive games and popular psychology have entirely different cultural origins. Elsewhere, neo-liberalism has been identified in the ideology of ‘linear time’ subtending European colonialism, thereby linking the critique of neo-liberalism to a strong theory of global- and post-coloniality (Shahjahan, 2015, p. 491). These strong theories present contrasting narratives about the historical duration and geographical scope of neo-liberalism, but they do share a commitment to distinguishing between neo-liberalism as a unitary cause and social interactions as derivative effects. This raises the following questions: when we use neo-liberal as a modifier – neo-liberal society, neo-liberal university, neo-liberal subject – do we mean that these phenomena would not exist without neo-liberalism? Or, do we mean that neo-liberalism has merely modified already existing phenomena?

This article argues that the strong theorization of neo-liberalism does not provide an adequate frame for understanding the local circumstances attending one of its most recognizable components: measurement. We do so by examining contemporary pressures on, and techniques employed in, higher degree research (HDR) programmes. It is an educational truism that the most exciting learning experiences are hard to measure, and that the experience of being measured is hardly exciting. The experience of completing a PhD dissertation brings this issue into sharp relief, as graduates frequently encounter a mismatch between the intellectual and emotional work required to complete a thesis, and the performance indicators used to sort applicants for academic positions, which may only have a tangential relationship to the substantive achievements of the thesis. But singular learning experiences are difficult to incorporate into staffmeetings, curriculum reviews or institution-wide policy initiatives. Measurement can therefore arrive as a way of ‘getting things done’ in finite decision-making environments, even when this involves drawing upon impoverished representations of lived experiences. Rather than understanding measurement practices according to the criteria of naturalism that would seek to represent the world ‘as it is’, we understand such practices as material signs with varying degrees of embeddedness in social and institutional worlds. In this context, the article argues that many critiques of neo-liberalism do not sufficiently advance alternative ways to think about the purposes of higher education, and that correspondingly, ‘neo-liberalization’ does not exhaustively explain the issues attending increased measurement and surveillance practices in HDR environments. We therefore want to focus on the worlds that measurements bring into being, and on the criticisms made of these worlds in contemporary sociological research on higher education.

Measurement is examined here from three distinct viewpoints. Firstly, we revisit contemporary studies of the neo-liberal university, and consider the high premium placed on measurement as the privileged technology of institutional progress. The article suggests that metrics often become mobile sites of powerful collective investment, and that even those sceptical about the purposes of metrics can come to care about and desire certain numbers. Secondly, we focus on specific concerns raised around measurement regarding HDR. Postgraduate environments are considered by many to be the final bastions of passionate intellectual inquiry protected from the continuous measurements that attend most undergraduate programmes. The incremental reframing of doctoral study in Australian universities as an outcomes-driven pursuit places heightened pressure on educators to clearly identify those features of long-term research projects that may have immeasurable outputs. Finally, as a gesture towards imagining alternative ways of thinking about learning outcomes,the article arguesthat pedagogical environments need to find waysto make HDR ‘failures’ – non-submissions, abandoned projects, flawed ideas – institutionally worthwhile and intellectually generative. It may be that measurement cultures are best inhabited by learning how to fail well.

Measurement and higher education

The relationship between quality and quantity has widespread significance for organizations interested in measuring ‘quality’ practices and objects (see Anderson, 2006). In simpleterms, qualities are properties that allow one thing to be distinguished from another thing. For example, blue is a quality that can be distinguished from red. Some qualities have direct quantitative correlates (blue has a wavelength of 450–495 nanometres), but many concepts describe non-quantifiable qualities. For example, the musical notion of ‘timbre’ has been used to describe composite variables, where no single quantity – amplitude, pitch, resonance – is completely essential to the concept. By contrast, quantification presupposes exactness in the units quantified, and there is no quantity that does not require a fixed separation between qualities. However, numerical measurements can easily become unmoored from the experiential world of qualitative differences, producing differential numbers with no sensible referent. For example, the birth ratein Australia in 2012 was 1.93 children per Australian female, and although qualities are required to distinguish one birth from another, one cannot imagine 1.93 children. Furthermore, a changing annual birth ratemay be detectable only across a large-scale population, and a national shift from 2.03 to 1.93 may go unnoticed within any particular community. Used in this way, measurement practices may risk alienating or disempowering those to whom they are applied, either because the objects being measured cease to be tangible in everyday experience (1.93 children), or because metrics may suffer from scaling effects, where extremely broad trends contain little explanatory power at a local level (2.03 to 1.93). The development of proxy indicators and probabilistic inferences (see Shachter, 1988) to measure large groups – students, workers, voters – can often produce measurements more useful for the governance of a population than for assisting the practices orimproving the well-being ofanyparticular individual within that population. This opens onto the issue of the organizational structures that coordinate measurement practices.

Within institutional settings, disagreements about the purposes, frequency and effects of measurement can reflect structural conflicts between various stakeholders in the production of quantitative data. Measurement practices simultaneously reflect and modify relationships between those who measure and those who are measured. In a variation of commodity fetishism, the aura of objectivity acquired by measurement practices can mask the social power relations between managers and the managed. This may also mean that key metrics come to be defined not by their utility to individual practitioners, but by their capacity to travel upwardly through an organizational administration. Numbers begin signifying more easily as reliable ‘proof’ once they circulate beyond an original ‘circle of belief’ – that is, those who actually produced the numbers (Kamuf, 2007, p. 257). Issues of this kind are regularly raised regarding the measurement of ‘learning’ and ‘research’ in the Australian tertiary sector, and such measurements have particular impacts on HDR students.

We can only offer a sample here of the measurement practices that attend HDR programmes in Australian universities, but they are broadly indicative of wider trends. Firstly, students are evaluated for entry into research programmes and eligibility for scholarships. For example, RMIT University’s ‘Model to determine merit-based selection’ establishes a candidate’s score based on their highest qualification (up to 55 percentage points), a school allocated score (up to 15), and either a combination of recent degree (20) and publications (10) or publications (10) and relevant professional experience (20) (RMIT, 2011, p. 1). In this formula, a Master’s degree by research is equivalent to 20 years’ relevant professional experience, and a refereed article in a scholarly journal is equivalent to an original creative work of international significance. Once enrolled, doctoral candidates are measured at regular intervals within their candidacy, often through discrete and ordinal indicators such as satisfactory, less than satisfactory, unsatisfactory. Although many doctoral candidates will not be able to pursue stable academic careers (see Mayhew, 2014), many will learn to recognize the KPIs used to measure academic performance, especially those that could shape future employment opportunities: publication outputs and the H-Index, student evaluation scores, ‘impact’ scores and so on. For the measurement of cohorts or populations, university administrations frequently mobilize recruitment rates, completion rates and post-doctoral employment rates. Differentiated metrics produce a logic of continuous improvement, where strong performance is not measured according to a fixed standard (60–70, 70–80), but is oriented instead towards infinite growth and expansion (e.g., 2% per annum).

Finally, any metric whatsoever can be subsumed within a system of ordinal measurements, or rankings. Ordinal rankingsfor programmes, institutions, journals and publishers can have a significant impact on what Guy Roberts-Holmes (2015) calls ‘data chains’, with institutions modifying postgraduate recruitment practices to improve the rank attached to programmes, faculties or entire universities, and to reduce any penalties attached to HDR candidates who fail to complete (this is relevant for the distribution of Australian Postgraduate Award scholarships). Unlike interval variables, ordinal variables do not require any specification of the gap between each placement, such that trivial differences between individuals can acquire a heightened sense of symbolic meaning (see Bowman & Bastedo, 2011, p. 441). Tacit logics of ranking may also shape the interpretation of data intended to be criteria-based. For example, the Excellence in Research for Australia (ERA) framework claims to evaluate programmes ‘against national and international benchmarks’, but the resulting numbers frequently circulate among potential postgraduate students as proxies for ‘best’ or ‘worst’ programme (on the ERA, see Redden, 2008).

Sociologists of higher education have amply shown that measurement practices across universities worldwide can produce distorted understandings of teaching, learning or research.¹ We want to focus less on criticisms concerned with true or false representations, and consider instead how measurement practices actively modify the worlds they seek to measure (see Burrows, 2012; Redden, 2015). As studies of standardized testing have shown, learning environments cannot be subject to continual measurement without bringing into being new professional practices and anxieties responsive to metric cultures (see Redden & Low, 2012). In some contexts, metrics used to determine teaching performance or research quality may even come to be treated as compliance-based games abstracted from pedagogical or intellectual purposes (see Anderson, 2006, p. 171). For this reason, in order to better understand forms of continuity and change in conceptions of HDR, the following section examines the overlapping critiques levelled at measurement under the concept of neo-liberalism. In the final section of this article, we link various effects of the forms of measurement that attend contemporary PhD programmes in Australia.

Four critiques of neo-liberalism in the university

Contemporary scholarship on measurement in higher education has been profoundly shaped by the concept of neo-liberalism (e.g., Ball, 2015; Gill, 2016; Redden, 2015). According to David Harvey’s influential approach, neo-liberalism holds that ‘the social good will be maximized by maximizing the reach and frequency of market transactions, and it seeks to bring all human action into the domain of the market’ (2007, p. 3). Harvey acknowledges that neo-liberal economic practices have developed ‘unevenly’ across the globe, and that state-sponsored neo-liberal policies have been ‘partial’, ‘lopsided’ and ‘tentative’ (p. 13). Nevertheless, despite the highly disparate phenomena now labelled as neo-liberal, Harvey suggests that the emergence of neo-liberal thought can be attributed to a broad historical situation: the crises of over-production in industrial and post-industrial economies that culminated in the early 1970s (pp. 11–14). This politico-economic explanation has faced numerous challenges, and Terry Flew (2014) notes that at least six distinct theories of neo-liberalism now circulate in sociology and economics. Among these, Michel Foucault’s lectures on neo-liberal techniques of government have become highly influential. Across his broad oeuvre,Foucault traces the development of state-based institutions – schools, hospitals, psychiatric wards, the police force – that bring forth subjective dispositions through direct intervention into everyday habits, routines, desires, anxieties and so on (see Donzelot, 2008; Foucault, 2008). Rather than describing a re-articulation between capital and labour, Foucault’s account of neo-liberalism focuses on its ethical prescriptions and epistemological presuppositions, including neo-liberal reconceptions of human behaviour, interest and justice. While Marxists and Foucauldians share an understanding that neo-liberalism seeks to marketize all manner of social rela...

Table of contents

• Cover
• Half Title
• Title
• Copyright
• Contents
• Citation Information
• Notes on Contributors
• Introduction - Academic life in the measured university: pleasures, paradoxes and politics
• 1. Higher degree research by numbers: beyond the critiques of neo-liberalism
• 2. The paradox of collaboration: a moral continuum
• 3. Measures of success: cruel optimism and the paradox of academic women's participation in Australian higher education
• 4. Counting on demographic equity to transform institutional cultures at historically white South African universities?
• 5. The mismeasure of academic labour
• 6. Rendering the paradoxes and pleasures of academic life: using images, poetry and drama to speak back to the measured university
• 7. Made to measure: early career academics in the Canadian university workplace
• 8. Fear and loathing in the academy? The role of emotion in response to an impact agenda in the UK and Australia
• 9. Challenging a measured university from an indigenous perspective: placing 'manaaki' at the heart of our professional development programme
• 10. Measuring the gift': epistemological and ontological differences between the academy and Indigenous Australia
• 11. Target-setting, early-career academic identities and the measurement culture of UK higher education
• 12. The missing measure? Academic identity and the induction process
• 13. Lost souls? The demoralization of academic labour in the measured university
• Index