The Impact of Higher Education Ranking Systems on Universities

Kevin Downing, Petrus Johannes Loock, Sarah Gravett

• Publisher: Routledge
• Year: 2021
• ISBN:
  9781000368109
• Edition: 1
• Topic: Education
• Subtopic: Education General

This book, written by three generations of rankings academics with considerable experience from three very different regions of the globe, lifts the lid on the real impact of higher education ranking systems (HERS) on universities and their stakeholders. It critically analyses the criteria that make up the 'Big Three' global ranking systems and, using interviews with senior administrators, academics and managers, discusses their impact on universities from four very different continents. Higher education continues to be dominated by a reputational hierarchy of institutions that sustains and is reinforced by HERS. Despite all the opinions and arguments about the legitimacy of the rankings as a construct, it seems experts agree that they are here to stay. The question, therefore, seems to be less about whether or not universities should be compared and ranked, but the manner in which this is undertaken. Delivering a fresh perspective on global rankings, this book summarizes the development of HERS and provides a critical evaluation of the effects of HERS on four different major regions – South Africa, the Arab region, South East Asia, and Australia. It will appeal to any academic, student, university administrator or governing body interested in or affected by global higher education ranking systems.

1 ‘Lies, damned lies and statistics’

Introduction

This chapter is perhaps controversially titled after a much-misquoted phrase, which seems particularly apt when describing the view many academics hold of Higher Education Ranking Systems or HERS for short. Perhaps the earliest citation of the phrase can be attributed to Arthur James Balfour, 1st Earl of Balfour, as quoted in the Manchester Guardian in 1892:

Professor [Joseph] Munro reminded him of an old saying which he rather reluctantly proposed, in that company, to repeat. It was to the effect that there were three gradations of inveracity – there were lies, there were damned lies, and there were statistics.

(Manchester Guardian, 29 June 1892)

HERS are often treated with contempt by many academics who are quick to point out methodological and other flaws (Altbach, 2006a; Dill & Soo, 2005; Downing, 2012). Even amongst academic supporters of rankings, HERS are treated with suspicion and criticised for a wide variety of reasons from the philosophical to the practical. The publication of ranking lists is now greeted with trepidation by many University Presidents and is often followed by intense questioning from the media who are interested to know what lies behind any annual rise or fall in rank on the global, regional or local stage. Competition between universities has undoubtedly intensified with the rise and expansion of Higher Education Ranking Systems (HERS), and many researchers agree that HERS, and the publication of annual rankings, has influenced all participating institutions to some extent (Espeland & Sauder, 2015; Hazelkorn & Ryan, 2013; Rauhvargers, 2014). The growing interest in HERS has sparked debate about the nature and validity of the various HERS and their methods (Altbach, 2006a; Dill & Soo, 2005; Downing, 2012). HERS have different parameters/indicators, including publication and citation counts, student/faculty ratios, percentage of international faculty and students, number of awards and achievements, number of research papers per faculty, web visibility and the number of articles published in high impact journals, to name but a few (Aguillo, Bar-Ilan, Levene & Ortega, 2010).

Controversy and criticism

Higher education has long been dominated by a reputational hierarchy of institutions that now sustains and reinforces HERS (Locke, 2014; Rauhvargers, 2014). Marginson (2007) points out that rankings reflect prestige and power, and rankings confirm, entrench and reproduce prestige and power. Rankings are criticised for many reasons including the use of mostly quantitative indicators, proxies to represent the quality of teaching and learning and the reliance on publications written in English (Kehm, 2014; Rauhvargers, 2014). Despite the range of opinions and arguments about the legitimacy of rankings as a construct, experts generally agree that they are here to stay (Downing, 2012; Hazelkorn, 2014). Therefore, the current question is less about whether universities should be compared and ranked, but the manner in which this is undertaken (Marope &Wells, 2013).

Scrutiny of HERS methodologies has increased considerably since 2009 (Baty, 2014). A Frequent criticism of HERS is that many ranking systems rely on poor indicators, such as reputational indicators, despite increased criticism from peers (Rauhvargers, 2014). The arts, humanities, and to a large extent the social sciences, remain underrepresented in rankings because of unreliable bibliometric data (Hazelkorn, 2013). Citation impact is still determined more reliably through indicators that measure the proportion of articles in intensively cited journals, and thus favours those fields in which these articles are concentrated, namely medicine, natural sciences and engineering (Waltman et al., 2011). Marginson (2007) argues that the measures of internationalisation some ranking systems employ are a better indicator of a university’s marketing function, rather than the international quality of its researchers. Student-to-faculty ratios are easily manipulated by institutions (Baty, 2014). The quality of teaching and learning and the ‘value added’ during the educational process eludes comparative measurement (Dill & Soo, 2005; Liu & Cheng, 2005). A lack of internationally standardised definitions makes it difficult to make valid comparisons across universities and countries (Waltman et al., 2011). Another problem according to Rozman and Marhl (2008) relates to the different cultural, economic, and historical contexts in which various higher education institutions function. Even some leaders of HERS acknowledge that universities and their characteristics can differ greatly, no matter where they are ranked in the various ranking systems (Sowter, 2013). Consequently, in particular at international level, there should be an awareness of possible biases, and the objectives of rankings have to be clearly defined. Scott (2013) elaborates on some other shortcomings of rankings methodology and identifies four key points:

• Ranking data are often used for other purposes like resource distribution.
• More generously funded institutions can attract students of higher quality and would, most probably, lead to higher employment ratings.
• A dearth of reliable data about the teaching (the primary function of the university).
• Ranking systems subjectively and deliberately attach weightings to the amount of relative worth of each ranking criterion.

Judgements and decisions based on university rankings should be made with knowledge and a clear understanding of the methodology utilised during the ranking process (Liu, 2013). Sowter (2013) admits that all ranking critiques have validity; however, HERS have contributed to transparency and accountability among institutions and contributed toward a culture of performance evaluation in higher education. Despite volumes of criticism and boycotts by some universities and schools, rankings have become a popular reference point for decision and policy makers (Hazelkorn, 2014). They have also produced their antithesis in the form of alternatives and have sparked a conversation about the role, value and contribution of higher education (Hazelkorn, 2014).

The positive side of HERS

Downing (2013) elaborates on some of the positive value of rankings and argues that prospective students and their parents can make informed decisions about institutions from a diverse set of global offerings. Prospective students, nationally and internationally, have access to information about the strengths and weaknesses of the institution, and of some departments within that institution (Downing, 2013; Sowter, 2013). Hazelkorn (2011) acknowledges that a good ranking position also makes it easier to attract international students. Institutional rankings can encourage university faculty to focus more effectively on the core business of higher education, teaching and learning, research and knowledge transfer and senior institutional managers can foster internal competition between departments to drive international competitiveness (Locke, 2014).

Another potentially valuable aspect of rankings is that it encourages the collection and publication of reliable national data related to respective higher education systems (Rauhvargers, 2014). Rankings are a useful benchmarking exercise for institutions and can help them make strategic decisions (Baty, 2014). They also serve as critical self-reflection tools whereby universities can use comparative citation information to enhance strategies to increase research quality processes (Downing, 2013). Universities can justify claims on resources based on improved ranking performance, and a high ranking will increase an institution’s ability to attract good partners and funders (Hazelkorn, 2011).

Younger institutions are now able to demonstrate to their governments, the higher education sector, funding bodies and the public that they have evolved or improved in certain areas (Downing, 2013). Industry makes use of ranking information to identify where to invest in higher education and innovation (Baty, 2014). Furthermore, Hazelkorn (2014) points out that HERS’ inability to accurately measure ‘quality’, exposed a deficit in higher education information sparking meaningful debates about the definition of higher education ‘quality’, ‘value’ and ‘impact’, and how these should be measured.

The case for consumers

Rankings already exert substantial influence on the long-term development of higher education across the world (Marginson & van der Wende, 2007) with three ranking systems currently in positions of relative global dominance. The oldest system is that prepared by the Shanghai Jiao Tong University (SJTU) which was first published in 2003 and is now known as the Academic Ranking of World Universities (ARWU). The QS World University Rankings published by Quacquarelli Symonds (QSWUR) was first published in 2004 alongside their partners Times Higher Education (THE). Around 2008 to 2009, this partnership ended and THE launched their own World University Ranking (THE WUR). These three rankings, often known as the ‘Big Three’, recognise the growing impact of the global environment on higher education systems and institutions and the importance placed on some means of identifying institutional excellence by prospective consumers. Some of these consumers have the advantage of government funded opportunities to access higher education, whilst many others will be spending their own money on obtaining the best education possible for themselves or, more likely, their offspring.

In almost every walk of life, we can make informed choices because we are provided with appropriate ways of assessing the quality of what we purchase and consequently narrowing down the choice of products we wish to investigate further. The advent of rankings has made it easier for prospective students and their families to access information about an institution that will assist with that choice. Whilst it does not always provide information about the strengths and weaknesses of the disciplines and departments encompassed within any given university, at undergraduate level it is often the reputation and ranking of the university that will encourage further investigation. In fact, outside of academic circles (and in some cases inside as well) the strengths and weaknesses of departments or disciplines within an institution are often ignored in favour of acknowledging that someone has a degree from a highly ranked institution. Academics, students, their parents and employers recognise this, and as students become more globally mobile, the reputation of any university, contributed to by its standing or ranking comparative to others, will continue to grow in importance.

We live in societies where competition is often regarded as a necessity in order to drive progress, and to continuously improve both the quality of products and the efficiency with which they are produced. Is higher education so different or remote from the real world that we are justified in arguing that we should not be subject to these universal forces? Of course not, in fact research has been driven by competition for hundreds of years and mankind has nonetheless managed to innovate and thrive. Rankings systems and criteria do encourage us to identify and engage in extensive benchmarking against institutions with a higher ranking than our own, providing some fascinating insights into how global peers tackle certain issues. Consequently, we can develop institutional systems which incorporate the best of benchmarked global practices, whilst ensuring these meet local requirements. This approach facilitates the identification of clear, agreed quantitative performance indicators for learning and teaching, globalisation and research which can be assessed at departmental level and within Colleges and Schools.

The impact of rankings on global higher education

Today over 60 countries have introduced national rankings, especially in emerging societies, and there are now a wide range of regional, specialist and professional rankings. What started as an academic exercise has now become a primary driver of a geopolitical reputation race (Hazelkorn, 2014). Rauhvargers (2014; 2013) lists a few policy implications which are a direct result of HERS influence:

• Immigration policies: For example, in the Netherlands, migrants who possess a degree from a higher education institution which is ranked in the top 200 have the privilege of obtaining ‘highly-skilled migrant’ status.
• Recognition of qualifications: The Russian Federation adopted Decision No. 389 which establishes an automatic recognition of qualifications issued by foreign HE institutions which are in the first 300 positions of the SRC’s ARWU, QS and THE rankings.
• Mergers are planned and underway in many European countries.
• Eligibility of Partner Institutions: In 2012, the University Grants Commission in India announced that foreign universities entering into bilateral programme agreeme...