Ratings and Rankings in Higher Education

A New-Materialist Exploration of How They Control Society

Jonas Thiel

• Publisher: Routledge
• Year: 2022
• Print ISBN:
  9780367417055
• eBook ISBN:
  9781000593808
• Topic: Psychology
• Subtopic: Education in Psychology
• Index: Psychology

This important work critically investigates the use of rating and ranking systems in higher education to show how they govern the academic population through the creation of competition and antagonism.

From social media to PISA and Rotten Tomatoes, ratings and rankings exist everywhere in our daily lives. Seemingly benign in practice, they can structure and govern important parts of society, including social interaction, public health and economic rankings. In this essential critique, author Jonas Thiel sets out the case against these practices, using the UK's higher education model to show how tools such as the National Student Surveys (NSS) instead divide the academic population to make it governable and controllable. Instead of achieving its intended aim of improving teaching by forcing competition over student satisfaction, Thiel shows that systems like the NSS have a profound and often negative impact upon how people and institutions understand themselves. Drawing on the new materialist theory of Karen Barad, Foucault's governmentality and Laclau's understanding of antagonism, the book raises an urgent need to respond to these boundary-drawing practices, especially in light of rising inequality and ecological collapse, and poses the question: can we even imagine a world without 'Top 10' rankings and 'out of 5' scores?

Engaging with current debates around 'value', tuition fees and the role of higher education in society, this is fascinating reading for advanced students and academics in psychology, education, sociology and philosophy.

1Introduction

DOI: 10.4324/9780367815806-1

Facebook, Uber, Airbnb, PISA, Moody’s, TripAdvisor, Rotten Tomatoes – ratings and rankings have infiltrated society. Businesses fear low ratings and desire good ones. A state of national emergency is announced when a country’s credit rating decreases. Countries are ranked regarding how many COVID deaths, hospitalisations, cases and vaccinations they have. People compare how many ‘likes’ they get for their Facebook statuses and ‘selfies’. In short, ratings and rankings change how people, organisations and countries think and act in the world.

This book critically investigates ratings and rankings. It does so by focusing primarily on the UK National Student Survey (NSS) where students score their universities, similar to a customer satisfaction rating at H&M. These ratings then influence ranking positions in university league tables. Universities hope that with good ranking positions, they may attract more tuition fee-paying students. By forcing competition over student feedback scores, so the story goes, universities will improve their teaching and offer better ‘value for money’. In this book, I, however, argue that this is untrue. Far from enhancing students’ education, the NSS functions in rather insidious ways: it divides the academic population and makes it hence governable and controllable. To theorise this is the mission of this book.

Yet, before digging deeper into the NSS, I would like to spend a little time exploring ratings and rankings more generally. I start by inviting you to consider the following four scenarios.

Example 1: My wife and I went on a trip to Rome. I remember that after visiting the ancient Roman Colosseum, we felt rather hungry and exhausted. Luckily, we found a cafe close by in one of the quaint side streets and decided, for once, not to consult our smartphones – we simply wanted to be ‘spontaneous’, relax and avoid becoming distracted by our phones. Excitedly, I ordered a ‘wild mushroom pasta’. After 5 minutes, I was presented with overcooked pasta with an unpleasant-smelling sauce, notably without any mushrooms. This meal, quite frankly, was barely edible. Afterwards, I asked myself the question what other people wrote about the place online, and it turned out that it was not enlisted on any of the mainstream apps, such as TripAdvisor or Google. From here onwards, we always checked on TripAdvisor before being ‘spontaneous’. As a result, we found a place in an excellent location and exactly at the price level we could afford; however, the restaurant was only rated at 3.1 stars. What did we do? We decided to walk further to another restaurant which scored 4.1. The food turned out to be mediocre. I’m not a snobby restaurant critic.

One may ask the question, ‘After you’ve told me this story, what is the problem with ratings and rankings? They are great for finding the right places. They reward good cafes with good ratings and punish bad ones with bad reviews’. You may be right. But now consider my second scenario.

Example 2: Fictional character Haleema has a business in a wealthy southern European region. She feels lucky that she only ever received 5-star reviews from customers on TripAdvisor. However, her friend, Luke, who manages one of the expensive hotels in this town, had not been that lucky. One day, a wealthy guest booked a range of rooms in this hotel. (Admittedly, you must be exceptionally wealthy to book anything in this European region.) Upon arrival, this guest abruptly told the reception that one of the rooms was not needed any longer and that he did not expect to pay for this room. Luke politely replied that receiving refunds when cancelling at such short notice was impossible. Disgruntled, the customer went to his hotel room. Just a few minutes later, the reception received a phone call from the customer: ‘I’ve seen a rat in my hotel room’, the guest said, ‘but I’m willing to let this pass if I do not have to pay for the cancelled room’. Luke did not agree to this as he reasoned that the rat never existed. The next day, the hotel in question received a 1-star rating on TripAdvisor. This rating was accompanied by a comment that the hotel was infested with rats which was clearly visible to everyone visiting TripAdvisor.

Example 3: A new trend that emerged in recent years are ‘reciprocal ratings’. A perfect example of these ratings is used by the company Uber. Uber offers taxi rides. You press a button on your phone, the taxi arrives, you get driven to your destination and you pay automatically – no cash needed. After your ride, you are asked to ‘Rate Your Driver’. You feel good about the drive and give your driver 5 out of 5 stars. A few minutes later, you realise that the driver has also rated you. To your horror, you realise that the driver has only given you 3 stars. Even worse, you realise that Uber ratings can be seen by all future Uber taxi drivers. Imagine the effect of this on you. You give your taxi driver 5 stars, your taxi driver gives you only 3 and, now, the next Uber driver may not actually choose you as a customer but another person who has a better overall score. It is no surprise that people have reported suffering from anxiety because of their Uber ratings (Hunt, 2016).

Example 4: These ‘reciprocal ratings’ are taken to the next level in Charlie Brooker’s ‘Nosedive’. Here, everyone rates everyone. Whether you buy a coffee, chat on the lift or swear in public, bystanders will be eager to reward you with 5 stars if they like you or punish you with 1 star if they don’t, simply by swiping their fingers over their phones. Each rating contributes to an overall score for each person, and augmented-reality contact lenses allow this score to appear ‘floating’ next to your head. Low average scores will deny people entry to their workplaces, high-quality rental cars, attractive housing and flights. A very low score even results in incarceration. ‘Nosedive’ is a science-fiction dystopia; yet, there are warning signs that similar technology is being developed. For example, there are prototypical experimental designs for contact lenses that augment reality (Bolton, 2016). Moreover, a social scoring system has been trialled in China where, recently, a school denied access to a child from parents who were considered ‘antisocial’ based on their ‘social citizen scores’ (Bisset, 2018).

The NSS

After introducing ratings and rankings more generally, I now turn to the main enquiry of this book: the NSS. First implemented in 2005 (Ipsos MORI, 2018), some scholars have described the NSS as a ‘national feedback survey’ (e.g. Ashby et al., 2011:5), whilst others call it a type of Student Evaluation of Teaching (SET) (Cheng and Marsh, 2010). My take is that the NSS is a simple customer satisfaction survey. It contains the following 27 questions which attempt to gather final-year students’ ‘satisfaction’ with their courses (Cheng and Marsh, 2010).

NATIONAL STUDENT SURVEY 2017 – CORE QUESTIONNAIRE

Scale:

• Definitely agree
• Mostly agree
• Neither agree nor disagree
• Mostly disagree
• Definitely disagree
• Not applicable

Questions:

The teaching on my course

• 1 Staff are good at explaining things.
• 2 Staff have made the subject interesting.
• 3 The course is intellectually stimulating.
• 4 My course has challenged me to achieve my best work.

Learning opportunities

• 5 My course has provided me with opportunities to explore ideas or concepts in depth.
• 6 My course has provided me with opportunities to bring information and ideas together from different topics.
• 7 My course has provided me with opportunities to apply what I have learnt.

Assessment and feedback

• 8 The criteria used in marking have been clear in advance.
• 9 Marking and assessment have been fair.
• 10 Feedback on my work has been timely.
• 11 I have received helpful comments on my work.

Academic support

• 12 I have been able to contact staff when I needed to.
• 13 I have received sufficient advice and guidance in relation to my course.
• 14 Good advice was available when I needed to make study choices on my course.

Organisation and management

• 15 The course is well organised and running smoothly.
• 16 The timetable works efficiently for me.
• 17 Any changes in the course or teaching have been communicated effectively.

Learning resources

• 18 The IT resources and facilities provided have supported my learning well.
• 19 The library resources (e.g. books, online services and learning spaces) have supported my learning well.
• 20 I have been able to access course-specific resources (e.g. equipment, facilities, software, collections) when I needed to.

Learning community

• 21 I feel part of a community of staff and students.
• 22 I have had the right opportunities to work with other students as part of my course.

Student voice

• 23 I have had the right opportunities to provide feedback on my course.
• 24 Staff value students’ views and opinions about the course.
• 25 It is clear how students’ feedback on the course has been acted on.
• 26 The students’ union (association or guild) effectively represents students’ academic interests.

Overall satisfaction

• 27 Overall, I am satisfied with the quality of the course.

(Ipsos MORI, 2017:online)

In 2018, the NSS was open for 16 weeks from January to April and all universities in the UK were supposed to participate (Cheng and Marsh, 2010). The UK government pays the multinational corporation Ipsos MORI to run the NSS. Ipsos MORI suggests that all responses in the survey ‘remain strictly anonymous’ (Ipsos MORI, 2018:online). There are similar student feedback questionnaires in other countries (e.g. Kane et al., 2008; Kuh, 2009).

To exemplify the NSS, imagine 21-year-old final-year undergraduate student Tracey. It is a cold January morning, and Tracey feels anxious as she knows that the final year counts. Tracey receives an email that tells her to complete the NSS and that she may even win a prize if she takes part. She sits down and, slightly grudgingly, begins the survey. The first question appears: ‘The teaching on my course – Question 1: Staff are good at explaining things’. Tracey selects ‘mostly agree’. The next question appears. ‘Staff have made the subject interesting’. Tracey clicks ‘mostly agree’. She begins to wonder, ‘Do they mean “staff who are teaching face to face” or “staff on podcasts”?’ She suddenly remembers a boring session on ‘How to conduct an interview’. She still clicks ‘mostly agree’. Tracey begins to ask how many more answers she needs to provide before she can continue writing her dissertation. After the 10th question, it becomes boring to fill in the NSS, and Tracey realises that she hardly reads the questions. She simply clicks ‘mostly agree’ because she needs to return to her dissertation writing.

This, of course, is a biased description as it captures how I approached these surveys as a student. Generally, I enjoyed my courses, so I answered all questions with ‘mostly agree’. There may be students who deliberate and reflect on their answers, carefully weighing up their degrees’ advantages and reflecting on whether it is fair or unfair to give certain experiences precedence over others. These reflections then culminate in 27 judgements that express an accurate and unbiased interpretation of three years of formative experience. Any confounding factors, such as recent relationship breakups, memorable parties with friends and disputes with landlords or credit-card companies, are rationally excluded from influencing the answers given. Perhaps such diligent students exist. Perhaps, on the other hand, some students simply fill in the NSS after a long night and answer every single question with ‘neither disagree nor agree’.

Why do universities allocate such ...

Table of contents

• Cover
• Half Title
• Series
• Title
• Copyright
• Contents
• Series Preface for Jonas Thiel’s Ratings and Rankings in Higher Education: A New-Materialist Exploration of How They Control Society
• 1 Introduction
• 2 Conceptual Foundations: Karen Barad’s Agential Realism
• 3 Ratings and Rankings as Apparatuses
• 4 The NSS as a Disciplinary and Neoliberal Hybrid
• 5 Divide and Rule: The NSS as an Antagonistic Governmentality
• 6 Governmental Apparatuses of Bodily Production
• 7 Conclusion – Solidarity, Accelerationism and Utopia
• References
• Index