Principles of Methodology

Research Design in Social Science

Perri 6, Christine Bellamy

• Éditeur: SAGE Publications Ltd
• Année: 2011
• ISBN:
  9781446291634
• Édition: 1
• Sujet: Sciences sociales
• Sous-sujet: Recherche scientifique et méthodologie

This book provides a comprehensive, accessible guide to social science methodology. In so doing, it establishes methodology as distinct from both methods and philosophy.

Most existing textbooks deal with methods, or sound ways of collecting and analysing data to generate findings. In contrast, this innovative book shows how an understanding of methodology allows us to design research so that findings can be used to answer interesting research questions and to build and test theories.

Most important things in social research (e.g., beliefs, institutions, interests, practices and social classes) cannot be observed directly. This book explains how empirical research can nevertheless be designed to make sound inferences about their nature, effects and significance.

The authors examine what counts as good description, explanation and interpretation, and how they can be achieved by striking intelligent trade-offs between competing design virtues.

Coverage includes:

• why methodology matters;

• what philosophical arguments show us about inference;

• competing virtues of good research design;

• purposes of theory, models and frameworks;

• forming researchable concepts and typologies;

• explaining and interpreting: inferring causation, meaning and significance; and

• combining explanation and interpretation.

The book is essential reading for new researchers faced with the practical challenge of designing research. Extensive examples and exercises are provided, based on the authors? long experience of teaching methodology to multi-disciplinary groups.

Perri 6 is Professor of Social Policy in the Graduate School in the College of Business, Law and Social Sciences at Nottingham Trent University.

Chris Bellamy is Emeritus Professor of Public Administration in the Graduate School, Nottingham Trent University.

PART I

Foundations

What methodology is and does

ONE

Inference and warrant in designing research

---

This chapter will:

• explain what is meant by methodology and how it differs from method;
• introduce the three main types of research question in social science, and how each is answered by drawing inferences from patterns found in data; and
• explain that methodology is always controversial, because all good things do not go together, and that trade-offs must be struck between the virtues of good research design.

---

How does methodology differ from the study of methods?

Since this book is about methodology, we should start with that term. But, first, we should point out that many standard textbooks use it loosely to refer to anything to do with research methods. So do not be surprised if you read books or articles that claim to be about methodology, but which deal with issues that are excluded from this book. Our definition, by contrast, is narrow and specific, and distinguishes clearly between method and methodology in social science.

In this book, we define method as the set of techniques recognised by most social scientists as being appropriate for the creation, collection, coding, organisation and analysis of data.

• Data creation methods are used to produce the raw material of research, namely wellstructured data – or sets of information – that can be used to perform further investigations, of the kind described below. Data creation methods include ethnographic or participant observation, focus groups, individual interviews, questionnaire surveys and so on.
• Data collection methods are procedures for capturing what is important for answering the research question from the data that have been created. They may involve scanning text for particular themes, codes or content or undertaking counts or more advanced quantitative procedures. However, we can only count or code once we have decided how to identify what is important, as we show in Example 1.1.

---

EXAMPLE 1.1. STREET LIFE

It is claimed that the number of people sleeping rough on the streets in British cities fell fairly sharply in the four or five years after the British Labour government’s initiative on rough sleeping in 1998. But it then levelled out, and at the time of writing appears to be increasing again.

However, as a consultation paper issued by the subsequent coalition government (Department of Communities and Local Government, 2010) shows, there is a major problem with this claim, in that no one believes that the data on rough sleeping are accurate, because counting the number of rough sleepers is far from straightforward.

The government is worried that its official definition of rough sleepers as ‘people sleeping or bedded down in the open air’ means that local councils do not count people who spend the night awake or sitting up in sleeping bags. But does it follow that councils should count all people on the street with sleeping bags? What, for example, about people who may use them as an aid to begging, but do not actually sleep rough? And should councils count people who sleep in tents, stairwells of blocks of flats or who take refuge on cold nights in shelters run by charities?

---

• Data coding methods are procedures for determining whether the information indicated by a particular datum or set of data meet the standards or thresholds required for them to be classified under a category, where that category is related to the research question or hypothesis.
• Data organisation methods are procedures for laying out whole sets or series of data, that have either been created, collected and coded by the researcher for the purposes of the project, or been taken from another source – for example, a national survey data set such as the British Crime Survey (BCS) or, as in Example 1.1, the British government’s annual estimate of rough sleepers (available at http://www.communities.gov.uk/publications/corporate/statistics/roughsleepingcount2010). Data organisation involves setting out the data on a suitably common basis – for example, by tabulating them – so that they can be analysed.
• Data analysis methods are procedures for manipulating data so that the research question can be answered, usually by identifying important patterns. Statistical procedures are obvious examples. There are many qualitative analysis techniques too, such as open-ended content analysis, and a variety of theory-based comparative techniques for handling historical qualitative data of the kind we shall discuss in Chapter 17.

---

EXERCISE 1.1. RIGOROUS BUT ROUGH

Taking account of the challenges identified in Example 1.1 above, think about how you could develop a consistent and accurate count of rough sleepers. What criteria would you use? How would you justify them?

---

Interpretation

You will notice that nothing has yet been said about the ‘interpretation’ of data. The process of data collection almost always requires the researcher to ‘interpret’ the data, and that this is particularly so when – as in Example 1.1 – the things being studied do not fall neatly into convenient, unambiguous units. We shall consider some of these issues in more detail when we discuss the use and application of concepts in Chapter 9. Coding likewise involves interpretation, because the decision whether the data indicate that a case meets standards for a particular code is an interpretive act of scientific judgement.

‘Interpretation’ is also required in the process of determining whether the data analysis supports the general conclusions drawn from the research, to answer the research question. We call this support, its warrant. Warrant is a central issue in methodology, and therefore one that will be addressed throughout this book.

A third meaning of ‘interpretation’ in methodology is discussed briefly below and will be discussed again in detail in Chapters 15 and 16. This third meaning of interpretation is restricted to particular kinds of data and particular sorts of conclusions – namely, those which attribute beliefs, ideas, emotions or ways of classifying to people being studied.

But the point to emphasise here is that all methodological approaches rely to a large extent on ‘data interpretation’ and therefore ‘interpretation’ is not a separate stage or activity from the ones we list above. Although research proposals are often written with timetables describing ‘data interpretation’ as if it were the final stage of a project when conclusions are to be drawn about the theoretical or practical significance of the research, in fact interpretation is at the heart of the whole research process.

So what is ‘methodology’?

The key lesson from this discussion is that methodology is not just – and is often not very much at all – a matter of method, in the sense of using appropriate techniques in the correct way. It is much more to do with how well we argue from the analyses of our data to draw and defend our conclusions. The methodological question posed by our rough sleepers example is just what would allow us to claim that an increase in rough sleeping has occurred; that is, to make inference to a description. If we went on to claim that a rise in rough sleeping is being caused by the economic downturn, then this would be an inference to an explanation. Or perhaps it would be illuminating to explore what rough sleepers themselves would count as rough sleeping and why. This would require an inference to an interpretation in the third sense discussed above.

Because methodology is about arguments that show warrant for inferences, it makes no sense to break down the study of methodology according to the different stages involved in the research process, in the way that we have just done above for methods. Rather, we shall distinguish in this book between different approaches to methodology, and discuss the strategies appropriate to these approaches. We shall begin this discussion later in this chapter, when we discuss the differences between research designed to lead, respectively, to description, explanation and interpretation. But we stress throughout the book that each of these approaches raises the same basic methodological question – how and how far can you argue from the particular data to the particular conclusions, or, to put it another way, what argument, if any, do these data actually support?

Being able to draw sound conclusions depends on designing all stages in the project on sound methodological principles. Conversely, it is entirely possible to follow prescribed methods carefully, but still produce methodologically suspect research, if the conclusions drawn from it are not soundly based. These problems are inescapably theoretical ones, because the study of methodology involves theories about how and how far the research design enables us to draw sound inferences to conclusions that provide answers to our research questions, or that determine how far our hypotheses are supported or undermined.

And that is what this book is all about.

Inference and warrant

The core concepts in methodology are those of inference and warrant, and we should explain here why they are so important.

We are used to opinion pollsters drawing conclusions about the voting preferences of over forty million electors by sampling the opinions of around a thousand people. They do this by using widely accepted principles of statistical inference. This example illustrates the problem that we often need to draw conclusions about a large population from what we can find out about a smaller sample. A second problem is that we cannot always observe the things we are interested in directly, but are forced to work with proxies or indicators. For example, psychologists make inferences about the working of human or animal brains from observing very fine movements of eyes. Industrial sociologists make inferences about organisational morale from the way workers behave or describe their feelings. And anthropologists interpret how human beings make sense of their worlds from their stories or other cultural artefacts. In none of these examples can synapses firing in brains, ‘morale’ or ‘sense-making’ be directly observed.

Furthermore, researchers could not confidently make inferences without theories – however implicit or provisional – about the relationships between the things in which they are interested and those things which they can directly observe. For example, using cultural artefacts to interpret sense-making depends on a theory of culture.

We can therefore define inference as (1) the process of making claims about one set of phenomena that cannot be directly observed (2) on the basis of what we know about a set of things that we have observed where (3) the choice of research instruments depends on a theory of how those instruments work.

We can define warrant as the degree of confidence that we have in an inference’s capability to deliver truths about the things we cannot observe directly. Warrant involves particular standards, which we shall discuss in more detail in subsequent chapters. We shall see, too, that some of these standards are more straightforwardly related to methods than others.

Observation

In the course of the book, we shall have occasion to use this slippery but absolutely unavoidable word in several ways. There are four different ways in which this word is used in social science methodology:

1. The value taken by a unit of data that is collected for, defined by and organised in a scheme of measurement. For example, the value ascribed to a variable entered into a cell on a spreadsheet or table is an observation on that variable. ‘Observation’ is used in this sense in the question, ‘what do the observations show?’
2. A unit of data, such as a case in a sample or data set, as in the question, ‘how many observations do you have?’
3. The systematic collection of data about behaviour or action, where the researcher cannot exercise experimental control over the regime of stimulus and constraint under which the research participants act, as in the term, ‘observational research’, which is the alternative to experimental research.
4. The activity of a researcher undertaking visual and/or audio inspection of participants’ behaviour, as in ‘a period of fieldwork observations’.

When we discuss some philosophical questions in Chapters 2, 3 and 4, we shall use the word in sense 1 a good deal. Chapter 5 considers observational research, in sense 3. In Chapter 6, when we discuss variable-oriented research, ‘observation’ will be used in sense 2. Although we bear sense 4 in mind throughout, it will come to the fore particularly in Chapters 15 and 16. You are warned to pause whenever you see the word, to make sure that you know what is meant. It will always be clear from the context which meaning is intended, but you can check either this page or the entry in the glossary if you need a reminder.

Some controversial claims about methodology

With those definitions in mind, it is time for us to make some big claims. Some are going to be controversial. You will find, as you read this book, that almost anything that is said in the field of methodology will attract disagreement. This is another big difference from the study of methods, because most people who study methods agree on what counts as, for example, transcribing an interview, or calculating a chi-squared test.

Here is our first big claim. Making warranted inferences is the whole point and the only point of doing social research, irrespective of what type of data and what style of research we use. The contribution to knowledge of any research consists in the inferences that can be made from it. Inferences are the principal products; they provide support for findings; and they are what make findings into findings rather than speculations, on the one hand, or raw data, on the other.

There are two reasons fo...