Analysing Quantitative Survey Data for Business and Management Students

Jeremy Dawson

• Publisher: SAGE Publications Ltd
• Year: 2016
• ISBN:
  9781473952836
• Edition: 1
• Topic: Business
• Subtopic: Ricerca e sviluppo
• Index: Business

In Analysing Quantitative Survey Data, Jeremy Dawson introduces you to the key elements of analysing quantitative survey data using classical test theory, the measurement theory that underlies the techniques described in the book. The methodological assumptions, basic components and strengths and limitations of this analysis are explained and with the help of illustrative examples, you are guided through how to conduct the key procedures involved, including reliability analysis, exploratory and confirmatory factor analysis. Ideal for Business and Management students reading for a Master's degree, each book in the series may also serve as reference books for doctoral students and faculty members interested in the method. Part of SAGE's Mastering Business Research Methodsseries, conceived and edited by Bill Lee, Mark N. K. Saunders and Vadake K. Narayanan and designed to supportresearchers by providing in-depth and practical guidance on using a chosen method of data collection or analysis.

1 An Introduction to Classical Test Theory and Quantitative Survey Data

1.1 About this Book

This book covers types of analysis that apply predominantly to data gathered via quantitative surveys. It is intended for non-experts who may be conducting survey research for the first time – for example for a student dissertation. As such, it has relatively few details of the mathematical underpinning of these methods (although some are essential to aid understanding), and concentrates more on the key principles of when and how the analysis should be done, and how it can be interpreted.

Although this text covers all of the key issues specific to analysis of quantitative survey data using classical test theory, many aspects of a broader study may be covered elsewhere. In particular, most types of analysis that would be used to describe data, or to test hypotheses, are covered in another book in this series (Scherbaum and Shockley, 2015). However, it will cover aspects of analysing survey data that other basic guides to data analysis might miss – in particular, data cleaning, reliability analysis, exploratory factor analysis (EFA) and confirmatory factor analysis (CFA).

Although it does not assume you have read a previous book in this series on questionnaires (Ekinci, 2015), it would be advisable to do this if you are starting by constructing and administering your own questionnaire as this book is concerned with data analysis rather than collection. There will be some points in this book where I refer back to that text rather than explaining something fully again.

1.2 Survey Data and Questionnaires

Surveys have long been used as an important method of data gathering in social science and other fields. As such, a wide range of methods have been developed over recent decades to enable the appropriate analysis of such data; in particular, in disciplines such as psychology this has become an important area of research in its own right (the field of psychometrics is devoted to studying the theory and technique of psychological measurement, which is predominantly survey-based).

One of the reasons surveys are so popular is their flexibility. Then can be used to collect both quantitative and qualitative data (this volume is concerned only with the quantitative part); they are sometimes the only valid way of collecting some quantitative data (e.g. opinions, attitudes, perceptions – things that cannot be measured directly); and they allow a large amount of data to be collected using consistent and relatively inexpensive methods.

Quantitative data within questionnaires can take on different forms, however – as it can with any source of data. Before going into detail on the types of methods that can be used, it is essential to understand these different data types.

The first distinction it is important to understand is that between categorical and numerical data. Categorical data refers to variables that can take on different categories – for example, sex, nationality and occupation. Each value represents a different thing, or category, but this is not a numerical quantity (although we may choose to use numbers as labels for these categories). Numerical data refers to variables that have a meaningful numerical value, whether on a naturally occurring or constructed scale – for example age, income, number of children, well-being measured on a scale from 1 to 10.

However, within these two major groups, there are sub-groups that it is important to understand: these are described and exemplified within Table 1.1.

Table 1.1

^*Of course this is not necessarily a simple binary variable as it fails to take into account transgender people – however, it is often measured in a binary way.

The distinction between these different types and sub-types is crucial for making decisions about analysis. Some types of analysis (e.g. correlations) only make sense with continuous data; some (e.g. chi-squared tests) only make sense with categorical data; some (e.g. one-way ANOVA) require a mixture of the two. However, these types of analysis are covered in Scherbaum and Shockley (2015), and will only be mentioned briefly in this book.

For the majority of this book we will concern ourselves with one particular type of data that is very common in questionnaires, but doesn’t fit neatly into the categorization in Table 1.1: Likert scales.

1.3 Likert Scales

Likert scales, named after the twentieth-century American psychologist Rensis Likert, are a method of measuring a variable (construct) that cannot be directly measured, by asking respondents to what extent they agree with a series of statements. Each statement is known as an ‘item’; technically, a Likert scale is the summation (or average) of the different items, although it is sometimes used to refer to an individual item as well.

For example, a set of three items relating to extraversion (Lang et al., 2011) is:

• I see myself as someone who is talkative.
• I see myself as someone who is outgoing, sociable.
• I see myself as someone who is reserved.

For each item, the respondent would typically choose from one of the following options:

• Strongly disagree.
• Disagree.
• Neither agree nor disagree.
• Agree.
• Strongly agree.

Sometimes a different set of response categories might be used: for example, options ranging from ‘Very dissatisfied’ to ‘Very satisfied’, or from ‘Not at all’ to ‘All the time’. Technically these should be referred to as Likert-type scales, as the original definition of Likert scales refers to items asking about the level of agreement only; however, it is common for all such scales to be referred to as Likert scales, and so this book will use the term ‘Likert scales’ to refer to all sets of items with such rating scales.

The number of response options may vary: in the above example there were five, but this may be four, six, seven or indeed any higher number. Note that if there are an odd number of items, then a symmetrical scale will yield a neutral or average item (e.g. ‘Neither agree nor disagree’) whereas an even number of items (e.g. six) will not.

In any case, each item should be considered as an ordinal variable. That is, the respondent chooses from one of a number of ordered categories. However, when taken together this changes. The purpose of asking three separate questions about extraversion here is not that the individual items are themselves of particular interest, but that between them they should give a better overall indication of the level of extraversion. Therefore a single score for the construct (extraversion) needs to be created.

For this purpose, a number is assigned to each of the responses – typically these would be 1, 2, 3, 4, 5 for the above example, although for the third question (which measures the extent to which someone is reserved – the opposite of what we would expect for an extravert) a response of ‘strongly disagree’ means higher extraversion, and therefore we would code these responses as 5, 4, 3, 2, 1 respectively. This is often referred to as a ‘negatively worded question’.,¹

Figure 1.1 Lang et al.’s (2011) extraversion scale in questionnaire form

The overall score for extraversion, then, would be calculated as the average – the arithmetic mean – or, alternatively, the sum of these three item scores. So, for example, if someone responded ‘Strongly agree’ to ‘I see myself as someone who is talkative’ this would be scored 5; if they responded ‘Neither agree nor disagree’ to ‘I see myself as someone who is outgoing, sociable’ this would be scored 3; and if they responded ‘Disagree’ to ‘I see myself as someone who is reserved’, this would be scored 4 (because it is a negatively worded question). The overall extraversion score for this individual, then, would be (5 + 3 + 4) ÷ 3 = 4.0.

You may notice that, by doing this, we are treating the ordinal measurement of the individual items in a more numerical way, and the eventual score for extraversion is no longer categorical, but actually resembles an interval variable. In fact, it is then usually treated in analysis as if it is a continuous, numerical variable.

Is this justified? Research suggests that it can be, but only if the Likert scale (or just ‘scale’) has good reliability and validity – concepts that are hugely important in survey research, and will be the focus for large sections of this book. These will be introduced formally in Chapter 2.

1.4 Classical Test Theory

Classical test theory (CTT) is the measurement theory that underlies the techniques being described in this book. It was developed by Mel Novick, who first published the codification in 1966 (Novick, 1966). It is based around the idea that any measured score consists of two parts – a true score and an error. The ‘error’ may represent measurement error, and other types of random or systematic error too – examples of these will be shown in the next chapter. As we will also see in Chapter 2, this can be expressed in a formal mathematical way that allows us to express concepts like reliability and validity in a more formal way.

A key concept in CTT is that, even though our measurement may be constrained by the tools we used, the underlying (true) score is on a continuum. Thus, when someone answers a question such as ‘I see myself as someone who is talkative’, their true perception of how talkative they are could fall anywhere within a given range – between the point where they would only answer ‘strongly disagree’ and the point where they would answer ‘strongly agree’. If they consider themselves to be quite talkative, for instance, but perhaps ...