Perspectives on Research Assessment in Architecture, Music and the Arts

Discussing Doctorateness

Fredrik Nilsson, Halina Dunin-Woyseth, Nel Janssens, Fredrik Nilsson, Halina Dunin-Woyseth, Nel Janssens

• Publisher: Routledge
• Year: 2017
• Print ISBN: 9781138695573
• eBook ISBN: 9781315526638
• Edition: 1
• Topic: Architecture
• Subtopic: Architecture General

Research in the creative fields of architecture, design, music and the arts has experienced dynamic development for over two decades. The research in these practice- and arts-based fields has become increasingly mature but has also led to various discussions on what constitutes doctoral proficiency in these fields. The term 'doctorateness' is often used when referring to the assessment of the production of doctoral research and the research competence of research students, but in architecture and the arts, the concept of doctorateness has not yet attained a clearly articulated definition. The assessment of quality has been practiced by way of supervising, mentoring and the evaluation of dissertations but much less discussed.

This book offers perspectives on how to qualify and assess research in architecture, music and the arts. It creates a broader arena for discussion on doctorateness by establishing a framework for its application to creative fields. The book is grouped into three sections and includes contributions from international experts in the various fields working in Australia, Belgium, Brazil, Canada, Norway, Sweden, Switzerland, Spain, the Netherlands and the UK. The first section offers general frameworks for further conceptualising doctorateness in the fields in question. It is followed by a section that describes and discusses various experiences, concerns and visions on the production and assessment of doctoral research reporting from doctoral programmes in different stages of development. The third section includes future-oriented perspectives on knowledge-building processes, and asks how the ongoing, profound changes in academia could influence the concept of quality in both doctoral process and product.

The book presents different perspectives on research assessment practices and developments of relevant criteria in the practice-based and creative fields of architecture and the arts. The contributions propose ways of framing this issue conceptually, show the need for awareness of the specific context and tradition programmes develop and give proposals for various potential trajectories for the future.

Section 1
Framing ‘doctorateness’

Doctorateness: where should we look for evidence?

Michael Biggs

Emerging epistemic communities and cultures of evidence: on the practice of assessment of research in the creative fields

Halina Dunin-Woyseth & Fredrik Nilsson

Setting the scene: the development of formal frameworks for doctorates in Europe

Anne Solberg

1
Doctorateness

Where should we look for evidence?

Michael Biggs

Discussion and debate about the nature and role of the doctorate, and therefore how it should be evaluated, can be found in all disciplines. However, there are certain disciplines in which the debate seems particularly wide ranging, and therefore the topic of evaluation has less consensus. In the creative arts, including music, architecture, creative writing, fine arts and design, and the like, the debate about doctoral research often includes fundamental issues about art and knowledge (Eisner 2008) and consequently what the creative arts can contribute to academic research (Knowles and Cole 2008). In education, the debate is often focussed on the transferable concept of level of achievement or competencies called ‘doctorateness’ (Traf-ford and Lesham 2009). I intend to approach the issue of the contribution to the arts of doctoral research by looking at a recent dispute in the educational debate about doctorateness. In this dispute, Wellington and Poole adopted different analyses of doctorateness, resulting in opposing conclusions.

Wellington offers five main arenas in which the concept of the doctorate can be discussed: the purposes of doctoral study, the impact of doctorates, written regulations for the award of the doctorate, the examination process and the voices of those involved in it (Wellington 2013, 1491). He assumes that what is being evaluated is the written thesis rather than the competencies of the candidate, and despite analysing various constituent categories in each arena he concludes ‘we should give up a search for some sort of “inner essence” of doctorateness’ (Wellington 2013, 1501). Wellington believes that ‘doctorateness’ is an ‘essentially contested concept’ and that it suffers from the same kind of indeterminacy as Wittgenstein’s concept of a game (Wittgenstein 1953, §66). His main positive conclusion is that being ‘publishable’ is ‘the single most necessary (though not on its own sufficient) quality that makes up a doctorate’ (Wellington 2013, 1502). Poole (2014), on the other hand, rejects this response to the polysemous nature of many of the terms used in the debate about doctorateness. He argues that by unpacking some of the plurality or ambiguity we may yet refine the concepts and reveal the causal variants between the disputants. For example, he finds many opposed assumptions made by Wellington and others, including whether it is the thesis or the candidate that is being evaluated, whether it is the process or the outcome, whether the examiner is acting as gatekeeper or community builder and where on the ‘cline of originality’ we require a successful thesis to lie (Poole 2014, 7).

The disagreement between Wellington and Poole reveals two different approaches to where one should look for evidence. Wellington’s interpretation is based on case studies of doctoral supervision, whereas Poole’s is based on a structural analysis of the doctorate as a process. Both agree that the outcome should be significant to the community but disagree whether this consists in making an individual ‘contribution’ (Wellington) or in meeting community-endorsed criteria for being ‘publishable’ that is assured through the process of peer review (Poole). These differences can be explained by looking more closely at the evidence that is being used and in particular by looking at the broader social context in which doctorates are evaluated.

Social authorisation

Our expectations of the nature of doctoral study do not arise out of the blue – advanced study of any kind is rooted in the requirements and expectations of individual disciplines and framed within overall notions of training, accreditation and qualification in institutional settings such as universities, training colleges, industry and so forth. The doctorate is differentiated from other levels of education by its requirement that the candidate contributes to the knowledge or understanding we already have in the subject. This places two obvious requirements on the doctoral candidate. The first is that they gain a thorough understanding of the subject (sometimes known as the literature review) so that they know what is the current state of knowledge and where an original contribution might be made (sometimes known as the gap analysis). The second is that the candidate has techniques (sometimes known as research methods) that will allow him or her to make a valid contribution that is accepted by the community and other stakeholders (Park 2007, 7f.). The second requirement is normally satisfied by what is known as discipline-specific research training that connects generic knowledge-production techniques to the specialist interests of particular academic or professional communities.

What we understand by ‘research training’ looks very different from one discipline to another. For example, in the hard sciences, research training is often training in specific techniques that constitute the industry standard for competent professional practice, such as ‘the leaching test’. In the humanities, there tends to be less standardisation about what are current professional competences, and so research training tends to involve exposure to different interpretational frameworks such as post-structuralism, critical theory and so forth. In the creative arts the situation is even more diverse than it is in the humanities. Here one finds that research training takes a wide variety of forms but still has the same role in a doctoral programme, that is, to make the candidate aware of the ways in which different intellectual and critical approaches to an issue change the responses that would be appropriate and the kind of artistic outcomes that would result. Thus as one moves from the hard sciences through humanities and social sciences into creative arts, one sees that the nature of research training changes from training in specific technical competencies into training in a particular intellectual or artistic stance. I believe that both professional competencies and critical rigour are necessary in all disciplines, and so perhaps it would be better to say that in the hard sciences the emphasis is on technical competencies with less emphasis on but still a presence of intellectual and critical awareness and that in creative arts the emphasis is the reverse. As a thought experiment one can imagine that a doctoral candidate in science who failed to exhibit technical competencies would probably fail, but equally a doctoral candidate in creative arts who only exhibited technical competencies would also probably fail.

This balance of intellectual and technical competencies relates to the values of the individual disciplines. I have claimed elsewhere that research training enables the candidate to make a valid contribution to knowledge and understanding in their discipline (Biggs and Büchler 2007). Part of the validity of this contribution consists in demonstrating that the candidate understands how knowledge is currently produced in the field. It is a professional competence that in the humanities one reaches for an intellectual position rather than a piece of laboratory equipment in order to gather or interpret data. By engaging in a legitimised professional activity the candidate demonstrates membership of the community. The candidate also demonstrates that he or she is able to understand the current state of knowledge in the field and to undertake a gap analysis precisely because the intellectual and professional framing of the discipline is an integral part of identifying the content of the discipline. We therefore have a virtuous circle containing the knowledge and understanding of the discipline, the training that is necessary for a doctoral candidate, gap analysis and contribution of the candidate that ultimately feeds back into the shared knowledge and understanding of the discipline. This fairly traditional structural description of research practice is equally applicable to the creative arts and sciences given a certain flexibility about the reference of each of the terms beyond their traditional uses, for example, ‘literature review’.

The virtuous circle of knowledge is endorsed and authorised by the professional community, and the broader intellectual context in which it is situated gives us a clue as to how we might deal with paradigm shifts or novel worldviews (Guba and Lincoln 2005). The difference between a paradigm shift and what is simply incompetent professional activity lies in the ability of the community to perceive an advantage in the new way of thinking. This may take time, and there is always a certain amount of intellectual inertia or resistance. However, a permanent failure to persuade is perhaps the indicator of incompetence rather than revolution. Artistic practices are sometimes called transgressive, but the difference between what is transgressive and what is madness also lies in the ability of the community to find value in what is presented. Indeed, what is madness is also a notion of society and collective agreement rather than having objective criteria. Park (2005, 196) claims that the doctorate, rather than being an objective construction, is a social construction, echoing a broader contemporary trend in philosophy of science (cf. Bloor 1991 [1976]) towards social rather than epistemic foundationalism.

This thought experiment with notions of professional competence and community legitimisation reveals the way in which our notions of what is normal, transgressive or madness are grounded in consent, particularly amongst persons who are societally authorised to ‘tell us what to think’. For example, in the art world, expert representatives of the community such as curators find value in some practices that are regarded as madness by the general public – the ‘institutional theory of art’ defines ‘what is art’ by what curators say is art. The content and boundaries of disciplines and intellectual practices are not fixed, and we can expect that doctoral candidates who are working at the highest levels of intellectual training and creative artists who normally work at the maximum reaches of innovation might find themselves working on sites of tension. These tensions will also be exacerbated because of the societal situation in which the establishment, for example, the university, is authorised to endorse a candidate as someone who understands the limits of the discipline and becomes qualified to extend those limits. This implies a new ‘institutional theory of artistic research’ that defines ‘what is artistic research’ by what academics say is artistic research, with its concomitant tensions between academia, the art world and society as a whole.

Professional and academic values: what to look for

The assessment process has three principal stakeholders: the assessor, the person or work being assessed and the context in which the assessment is used – for example within an institution or within a social group who recognise the validity of that judgement. This social context may authorise the assessor to make judgements on behalf of the community, for example, we authorise certain assessors to make judgements about the competence of medical practitioners on our behalf, and we accept the consequence that some people are therefore allowed to practice medicine while others are not. Whether at a micro or macro level this process of assessment is normally undertaken by one group on behalf of and with the consent of another. At the moment, doctoral evaluation in the arts often lacks representation from the professional art world, reflecting the perception that artistic doctorates are relevant in academia but not necessarily in the gallery. As a result the dominant concept of doctorateness in creative arts constitutes an ‘institutional theory of artistic research’. I was once in discussion with a Swedish curator who said to me that she would only be interested in doctoral research in the arts when it produced good art, which revealed an erroneous assumption on her part that the institutional theories of art and artistic research are the same.

The difference between these two theories lies in the difference between professional and academic values in the arts and whether the professional objectives of curators and gallery owners are the same as the academic objectives of doctoral programmes or advanced training in the arts. There is no intrinsic reason these objectives should be the same, although there may be political reasons academic objectives should be brought more in line with professional objectives, for example in order to demonstrate that taxpayer/stakeholder investment is meeting real-world needs. The academic structure of research in any discipline does not necessarily result in commercial benefit, although it may so result. The commercial exploitation of knowledge in the form of R&D is often a separate process to the generation of theoretical or academic knowledge (by which I do not intend to imply any hierarchy of values). So it is that a process that satisfies the academic requirements for a doctorate or of doctoral training in the arts does not necessarily result in good gallery art. Similarly, good art does not necessarily warrant the award of a doctorate. The institutional theories of art and artistic research are not the same principally because those who are authorised by society to make value judgements are not the same, that is, curators on one hand and academics on the other.

Analysis of sources of data: where to look

Depending on the country in which the doctoral examination takes place, the evidence for awarding a doctorate may be intrinsic or extrinsic. Evidence that I describe as intrinsic would include the written thesis and any portfolio of artwork or previous publications that may constitute the formal submission. In Australia this is normally all that the examiner has as evidence, whereas in Europe the examiner is also presented with the candidate at a viva voce examination. Evidence that I describe as extrinsic would include all those extra-textual references that are not included in the submission, and the academic and societal context to which the study refers and against which it must be evaluated. In addition to differentiating intrinsic and extrinsic evidence we can differentiate between the generic and the discipline-specific aspects of the work. At a generic level we have the requirement that the doctoral candidate has been trained in some way as a researcher. At a discipline-specific level we have a requirement that the candidate or the study makes a contribution to the discipline. These four sources of data form a Boolean square within which we may consider the contested term ‘doctorateness’ (see Table 1.1).

The extrinsic-generic quadrant comprises the social context in which we have a class of persons with doctoral qualifications who normally pursue careers as researchers. The examiner comes from this class, and whether she is acting as a gatekeeper or a community builder, society authorises her to make judgements on its behalf. These judgements may have consequences that permit the candidate to work as a professional in sensitive areas such as medicine or managing the national economy. It is therefore a position of trust. As a society we have developed a construct in which going to university and completing a doctorate in some way equips the candidate to have the competencies that we demand in order to undertake certain roles. Whether this is actually a training or merely a rite of passage is a meta-le...

Table of contents

• Cover
• Title
• Copyright
• Contents
• Acknowledgements
• Editorial: the art of assessment – focusing research assessment from different perspectives
• List of contributors
• SECTION 1 Framing ‘doctorateness’
• SECTION 2 Various experiences, cases and concerns
• SECTION 3 Doctorateness to come?
• Index