External Quality Audit

Has It Improved Quality Assurance in Universities?

Mahsood Shah, Chenicheri Sid Nair

• Publisher: Chandos Publishing
• Year: 2013
• Print ISBN: 9781843346760
• eBook ISBN: 9781780633169
• Topic: Education
• Subtopic: Education Administration

Whilst external quality audits have been in place for more than a decade in some countries, limited research exists on the extent to which such audits have been effective in improving systems and processes for quality assurance in higher education institutions, and the extent to which such audits have improved academic standards, outcomes and student experience. External Quality Audit looks at the experience of countries where external quality audits have been established by governments, and provides analyses of their effectiveness in improving quality assurance in universities and other higher education institutions. - Brings together the experience of academics and administrators in higher education institutions - Examines international experience on the extent to which external quality audits have improved quality assurance at national, institutional and faculty levels - Gives perspective from a variety of higher education institutions

1

Quality assurance – corporate fist in a velvet glove?

Robin McTaggart

Abstract:

Quality assurance in Australia has been favourably received in the university sector because of the approach used by the Australian Universities Quality Agency (AUQA) since its inception in 2000. The approach balanced community, business and academic professional interests and supported many significant improvements in university practice, notably in concert with other government agencies. Teaching was an area where AUQA was influential, but causation is difficult to isolate because several things were happening at once. Shifts in government policy have weakened the role of quality assurance and replaced it with government attempts at business-style corporate management and monitoring together with reductionist ways of informing both policy and publics. This is inconsistent with the professional review that is at the core of disciplined academic life.

Key words

quality assurance

key performance indicators

government managerialism

control and diversity

standardisation and quality

Australian quality assurance

Proponents of the establishment of the Australian Universities Quality Agency (AUQA) in 2000 justified it in a discourse of globalisation and commercialisation: the establishment of Australian universities on- and off-shore as a destination for fee-paying international students. Australia was the only Organisation for Economic Cooperation and Development (OECD) country that did not have a national quality assurance system. Proponents argued that the absence of a nationally regulated quality assurance system would be a significant impediment to participation in the growing international market for higher education. In Australia, higher education was becoming a very significant export industry and government was concerned that the system was quality assured so as to overcome the anxiety that just one blemish in a university’s practice could have serious economic effects. The establishment of AUQA signalled emergent thinking about commercialisation, and within a few years universities and the growing private higher education sector were trading as if under ‘brand Australia’ and urging each other to protect the national brand that was originally conceived to revive the Australian tourism industry.

There were educational reasons for government and university action, too. Genuine concern was raised that the student experience of both domestic and international students was not as good as it might be. Several factors militated against the quality of teaching and student support. Large numbers of students were forced to work part time in order to support themselves, often for long hours. In some instances, the curriculum and timing of classes were adapted to the needs of students. This increased the complexity of work for professors. The influx of undergraduate international students, often not using English as their first language, added to these demands. Increased complexity, coupled with extension of workloads in most dimensions of teaching, created unforeseen demands on professors, many of whom were using lecture–tutorial ‘sage on the stage’ approaches to pedagogy, an inflexible approach in the light of new educational technologies. Concerns about teaching standards and support for students in this new environment were presenting as an issue for quality assurance.

Teaching was not the only escalating demand on professors. Pressure to improve the quality and quantity of research outputs was also rising. Other pressures were increasing too. For example, community engagement, in its numerous guises, was increasing in popularity as universities sought to establish greater community awareness of their value as a public good, and also as a way of increasing influence on government funding allocations. Unfortunately, university education ran a poor third behind schooling and vocational training in government budget allocations to education. However, the chief competitors for professors’ time, understandably, were research and teaching, with expectations and aspirations about each increasing.

Government advocacy and the perceived threat that research funding would be more focused, and distributed on the basis of previous performance, energised the research efforts of staff. More precise measurement of research performance and growing sector preoccupation with global measures such as the Shanghai Jiao Tong Index had two main effects: more internal effort to improve individual staff research performance and the shifting of funds into areas with better prospects of impact on research performance indicators. The reward systems of institutions were not really focused on quality undergraduate or postgraduate coursework teaching. Some external impetus was needed in order to improve and coordinate quality assurance practices, especially with respect to teaching. In short, it was decided by state and federal governments that here was a need for AUQA.

Measuring teaching quality

Government already had a range of performance monitors for the distribution (and redistribution) of research funds (and research students). The indicators used remained contentious, not least because their biases shifted research funds towards the natural sciences and to rather narrow concentrations of researchers. Nevertheless, it required no leap of bureaucratic imagination to try to create straightforward measures of teaching quality – a more difficult concept to describe. The goal was to establish a competitive pool of extra money, the Learning and Teaching Performance Fund (LTPF), to reward university performance in teaching. Vice-chancellors, now often called ‘CEO’ or ‘president’, seemed eager to do this. It obviously simplified the parameters of their own performance indicators, nurtured intra-national competition, and attractively promised universities (winners, that is) additional government funds at a time when per student capita funds were routinely declining.

What were appropriate measures of quality teaching? Already government had its own measures – more measures of ‘system performance’ than understanding much about quality teaching. The focus was basically return on investment, number of graduates per dollar of government funding. These included variables such as timely completion rates, retention, pass rates, and rates of transition from first to second year – poor transition was considered a major ‘waste’ factor. These parameters were precisely defined and were adjusted to take account of different factors that government-commissioned research had shown to influence student performance: intake score, socio-economic status, and distance from major city, for example. Student feedback was also to play a key role.

There were already student feedback scales previously constructed with the intention of providing information for prospective students – the Course Evaluation Questionnaire (CEQ) and the Graduate Destination Survey (GDS). They were not intended as measures of teaching quality, or for competitive evaluation and ranking of universities, either as single scales or in concert with other measures. There was no assessment of their appropriateness for these purposes, but they were quickly called upon for the purposes of allocation of the LTPF. The reasons were crudely pragmatic and commercial. In a sector ‘consultancy’ meeting about the proposed LTPF one vice-chancellor designate still to take up the position advocated: ‘We won’t get this money from government if we don’t do something. Let’s get the teaching performance system in place and worry about the validity of the indicators later.’

The LTPF was implemented with patchy support. It produced puzzling results because the correction factors turned out to be major influences on the rankings. Score differences between universities were very small and it became evident that large sums of money were being distributed on weak premises. Universities that had scored marginally less than others found it difficult to discern just what their staff development programmes might do to achieve improvement. Even if the small differences were real, it was a moot point whether funds should be redistributed on this basis at all. It was a case of statistical significance not being matched by educational significance, a phenomenon noted elsewhere (Cheng and Marsh, 2010).

The newly funded Australian Learning and Teaching Council (ALTC), established to improve university teaching by providing professional development, could be called upon to help those universities that did not measure up on the LTPF to improve their teaching. It did this to a degree, but its focus was developing highly regarded programmes for staff development for genuinely perceived educational problems. The ALTC summarised its role as follows:

The Australian Learning and Teaching Council is dedicated to improving the student learning experience by supporting quality teaching and practice. We work with eligible higher education institutions, discipline groups and individuals as a collaborative and supportive partner in change, providing access to a network of knowledge, ideas and people.

We support outstanding teaching and practice through a suite of award, fellowship and grant schemes. We provide funding opportunities through a major competitive grants scheme for innovation in learning and teaching. We also support outstanding scholars to undertake leadership activities through our fellowship scheme and coordinate the Australian Awards for University Teaching. The prestigious Prime Minister’s Award for the Australian University Teacher of the Year is part of the awards programme (http://www.altc.edu.au/who-we-are).

Opportunities for staff development for teaching were funded, and valued change occurred. ALTC had to some degree become the ‘front end’ of system quality assurance for teaching, but more importantly it helped to validate the work of teaching and learning development staff in the universities.

During the advent of AUQA, the government already had the skeleton of a performance management system in place. It was obviously not quality assurance in any comprehensive sense. Nevertheless, there was hope that AUQA could help universities to bring many such activities together by using a well-informed process of ‘peer review’ with international, business and community credibility through the composition of its Audit Panels. The standard composition was:

two members from within Australian universities;

an Australian member from outside the university sector (with highlevel experience and understanding of planning and quality assurance in industry or commerce);

an overseas member (usually an academic); and

an AUQA professional staff member (the Director or an Academic Audit Officer).

AUQA could provide legitimate public accountability and academically sound advice. Because of its approach AUQA also had the potential to perform a leavening role against the harsh statistical simplicities of the performance indicators increasingly sought by government in order to more directly manage the universities as corporate entities, with little differentiation between them and commercial corporations. The initial stance taken by AUQA was to focus on reporting the adequacy of the quality assurance practices of each university, with the expec...

Table of contents

• Cover image
• Title page
• Table of Contents
• Copyright
• List of figures and tables
• Preface
• About the editors and contributors
• Chapter 1: Quality assurance – corporate fist in a velvet glove?
• Chapter 2: The impact of external quality audit in a private for-profit tertiary education institution
• Chapter 3: Multiple views of quality audits in New Zealand: a case for advancing systemic understanding
• Chapter 4: Reflections on the effectiveness of four cycles of external quality audits in New Zealand universities
• Chapter 5: Enhancing quality assurance in an institution: the University of the South Pacific experience of an external audit
• Chapter 6: Quality audits in Hong Kong: seeking evidence of quality enhancement
• Chapter 7: External quality audits and quality assurance in the higher education sector: the Malaysian perspective
• Chapter 8: External quality audit in the UK: an impact analysis
• Chapter 9: Effect of external quality agencies on universities in India
• Chapter 10: External quality audits in South African higher education: goals, outcomes and challenges
• Chapter 11: Development, effectiveness and future of quality assurance and assessment in Italian higher education
• Chapter 12: Audits of quality assurance systems of higher education institutions in Finland
• Chapter 13: External quality auditing: strengths and shortcomings in the audit process
• Chapter 14: Institutional management and quality audit: the experience in Chile
• Chapter 15: Scope, transparency and style: system-level quality strategies and the student experience in Australia
• Chapter 16: Accreditation and institutional learning: the impact interactions based on a minimaxing strategy have on the benefits from external reviews
• Chapter 17: Has it worked globally?
• Index