Mixed Methods Research in Poverty and Vulnerability

Sharing Ideas and Learning Lessons

Keetie Roelen,Laura Camfield

• Publisher: Palgrave Macmillan
• Year: 2015
• ISBN:
  9781137452511
• Topic: Social Sciences
• Subtopic: Global Development Studies
• Index: Social Sciences

The added value of mixed methods research in poverty and vulnerability is now widely established. Nevertheless, gaps and challenges remain. This volume shares experiences from research in developed and developing country contexts on how mixed methods approaches can make research more credible, usable and responsive to complexity.

1

Introduction

Keetie Roelen and Laura Camfield

Background

The use of mixed methods in researching poverty and vulnerability and evaluation of interventions in this field has expanded rapidly in the last few years. The added value of mixed methods research in analysing poverty and vulnerability has now been widely acknowledged (see Shaffer 2013, Stern et al. 2012). Much work has been undertaken with respect to meaningfully combining methods at various stages in the research process – from generating data to analysis and reporting – and reflections thereon have led to mixed methods not only having become more ‘mainstream’ but also more robust and of greater quality. Despite an exponential growth of studies using mixed methods research in the last decade, gaps and challenges remain.

A workshop on mixed methods research in poverty and vulnerability held in London in July 2013 brought together academics, practitioners and consultants from developing and developed countries to share ideas and learn lessons about the use of mixed methods approaches in this particular area of study. A number of themes emerged in terms of where more advances are to be made, namely credibility, complexity and usability. This edited volume provides reflections on various issues within these themes, largely based on practical applications in research and evaluation. The collection includes contributions from different disciplinary perspectives and holds considerations on the process of data collection as well as the use of data for analytical and policy purposes.

In this introduction, we will discuss each of the three emerging themes and how they are covered in the contributions in this volume.

Credibility

Although mixed methods research in poverty and vulnerability may have firmly established itself as a valuable contribution to development studies, it still lacks credibility in many areas of academia. This holds particularly true for academics studying poverty and vulnerability from a singular disciplinary perspective such as economics (Shaffer 2013). Underlying this scepticism might be the epistemological clashes when trying to combine data and methods grounded in different disciplinary backgrounds. The field of impact evaluation has been particularly liable to such a divide, where quantitatively focused ‘randomistas’ often find themselves on the opposite side of heterodox quantitative, qualitative and Participatory Rural Appraisal (PRA)-influenced researchers (Bamberger et al. 2010). There may also be concerns relating to the rigour of mixed methods research given that few people are expert in both qualitative and quantitative data generation and analysis. This renders conventional guidelines for assessing quality insufficient. For example, Camfield (2014) notes that the mixing of methods requires an engagement with the metanarratives’ underlying assumptions about the topic under investigation and, therefore, with the epistemological understandings that shape those assumptions.

Considerations for improving credibility in mixed methods research, as well as the ways in which mixed methods approaches can make research more credible, are central to many contributions in this volume. From an epistemological perspective, a more explicit consideration of how different disciplinary backgrounds enter mixed methods approaches and shape the subsequent research design allows the researcher to extend beyond the implicit assumptions and methodological choices that are rooted in such disciplinary backgrounds. At the same time, greater reflection on disciplinary considerations that feed into the design of mixed methods approaches may allow users of research to overcome their own epistemological qualms. Edmiston (Chapter 3, this volume) shows how distinct citizenship theories and concepts of relative deprivation can be meaningfully and credibly combined through the study of lived experience, furthering our understanding of poverty and vulnerability in light of social, economic and cultural relations. Fahmy, Sutton and Pemberton (Chapter 2, this volume) highlight how consensus about ‘necessities of life’ is interpreted differently from quantitative and qualitative perspectives and that more deliberative methods are required for understanding public views on necessities.

Methodological opportunities for making analysis and presentation of findings more robust across a spread of methods, grounded in different disciplinary backgrounds, includes the role of methodological bilingualism by ensuring that the research team has experts from each of these backgrounds (Torres Penagos and Bautista Hernández, Chapter 8, this volume). The importance of combined use of methods and assigning equal weight to such methods is also considered imperative in overcoming epistemological and methodological divides and for adding credibility to the overall findings (Dawson, Chapter 4; Torres Penagos and Bautista Hernández). Finally, an issue often overlooked yet crucial for collecting credible and high-quality data is that of the positionality of researchers and the research-respondent relationships (Dawson). With respect to impact evaluation, Copestake and Remnant (Chapter 6, this volume) consider issues that tend to undermine its credibility, including the challenges of attribution and establishing external validity and systematic biases such as confirmation and pro-project bias. They conclude that greater emphasis on qualitative methods and the use of mixed method approaches might be most appropriate in addressing such issues.

Complexity

The use of mixed methods in research on poverty and vulnerability grounded in complexity frameworks is limited. This is despite growing recognition that pathways out of poverty are anything but linear, forcing us to think beyond direct impacts from single interventions and acknowledge ‘the multiplicity of contributions to development outcomes’ (Befani et al. 2014, p. 3). In a longitudinal mixed methods study in Bangladesh, Davis and Baulch (2011) found that household wealth can follow a range of trajectories, most of which are non-linear. Similarly, in a longitudinal study of 20 Ethiopian communities, the use of case-based methods for investigating changes over time showed communities to be ‘dynamic open complex systems’ (Bevan 2014).

Yet many studies appear to adopt the view of livelihood systems being clearly demarcated and delineated structures, and of poverty dynamics being linear processes. Evaluation studies appear particularly prone to such over-simplification. But as indicated by Picciotto (2014), while experimental impact evaluations may be able to attribute impact to an intervention, they are not able to answer questions about whether the intervention was appropriate, relevant or efficient. Pradel et al. (2013) argue that an outcome evaluation approach – focusing on proximate outcomes rather than impacts – is better suited to reflect both the complex contexts in which interventions take place and the many factors that lead to change. Mixed methods research can make an important contribution to studies that analyse poverty and vulnerability as complex and overlapping states, as opposed to delineated and linear processes. This includes evaluations of programmes and their contributions to poverty reduction and improved livelihoods.

The notion of complexity and the role of mixed methods in addressing such complexity is a key theme in contributions throughout this volume. Copestake and Remnant argue for the pursuit of realism in impact evaluations and therefore for a more balanced integration of methods, realising that confounding factors in such evaluations are too plentiful and change too rapidly for purely experimental quantitative evaluation designs. The contribution by Devereux and Roelen (Chapter 7, this volume) is based on precisely this premise: that programme impacts are non-linear, particularly when considering social dynamics and ‘true impacts’ over a longer period of time. They argue that mixed methods approaches are crucial in unpacking that complexity.

Relationships form an important element of this complexity. The importance of relationships as an inherent but often overlooked factor in understanding poverty and wellbeing, and the role for mixed methods approaches in unpacking such relationships, is emphasised in several contributions in this volume. Edmiston reveals how an integrated study of deprivation and citizenship arrangements by combining quantitative data on objective and subjective measures of deprivation with lived experiences is crucial for unpacking the complex dynamics of deprivation at the micro-level within the context of macro-level socio-economic relations. McGregor, Camfield and Coulthard (Chapter 10, this volume) argue for the importance of using human wellbeing as a measure of development, partly on the premise that relationships are core to human wellbeing and a neglect of this dimension would obscure the complexities underlying the process of development. The use of mixed methods is considered vital for moving beyond simplistic and static understandings of wellbeing and thereby development.

Usability

Despite the additional insight and texture that mixed methods studies offer to the issues of poverty and vulnerability, policymakers often remain sceptical of such studies (for reasons discussed above) (Shaffer 2013) and subsequently make limited use of them. However, as pointed out by Sorde Marti and Mertens (2014), social scientists not only have a responsibility to identify problems and provide insight into them and the processes leading up to them, but also to offer suggestions on how to respond to or solve those problems – to work towards ‘transformative social change’. Usability of mixed methods studies could be increased by introducing action research elements or by responding more directly to the information needs of policymakers when choosing methods and presenting findings.

Various chapters in this volume show how mixed methods studies can be user-friendly and meaningfully contribute to scientific and policy debates. Dawson shows how ‘conventional poverty measures’ provide a picture of development that may not necessarily resonate with those experiencing the effects of these policies. The mixed methods study juxtaposes findings following ‘conventional poverty measures’ and people’s own perceptions. Clear reference to information from both types of data and their contrasting insights makes the study more policy amenable. Torres Penagos and Bautista Hernández illustrate how the integration of data and methods at the municipal level generates information that is relevant for policymakers at that level, thereby facilitating policy uptake and shortening the linkages between evidence and policy impact. Finally, Burrows and Read (Chapter 9, this volume) discuss how an organisation-wide evaluation protocol can ensure that findings from country-specific mixed methods studies lead to greater policy uptake by the organisation’s managers.

This volume

As the three themes discussed above appear across the different chapters, this volume is structured around three main topics of study, clustering chapters into (i) poverty measurement, (ii) evaluation research and (iii) from research to policy. Within these three sections, individual chapters link to the knowledge gaps and challenges with respect to mixed methods research in poverty and vulnerability as discussed above. Contributions present case studies from developed and developing country contexts and applications of different approaches to mixed methods research, offering substantive findings and reflections following their use.

Section I pertains to studies regarding poverty measurement, including how mixed methods research can contribute to interpreting measures of ‘necessities of life’ in the United Kingdom (Fahmy, Sutton and Pemberton, Chapter 2), understanding deprivation and social citizenship in the UK (Edmiston, Chapter 3), contrasting pictures of development and poverty reduction in Rwanda (Dawson, Chapter 4), and vulnerability and resilience in Burkina Faso (Tincani and Poole, Chapter 5). Section II offers ref...