Repositioning Out-of-School Learning

Methodological Challenges and Possibilities for Researching Learning Beyond School

Jo Rose, Tim Jay, Janet Goodall, Laura Mazzoli Smith, Liz Todd, Jo Rose, Tim Jay, Janet Goodall, Laura Mazzoli Smith, Liz Todd

• Publisher: Emerald Publishing Limited
• Year: 2022
• Print ISBN: 9781787697409
• eBook ISBN: 9781787697416
• Topic: Education
• Subtopic: Education Theory & Practice

Out-of-school learning spans varied formal and informal contexts and is hugely important for the lives of children. The need for time, flexibility, and agility in research within this field is highlighted throughout this multi-disciplinary edited volume, as each author reflects on how to make sense of the unknown and varied contexts in which out-of-school learning takes place. A range of different case studies discuss research methods used, challenges faced, and ways challenges were overcome in relation to out-of-school learning are presented, followed by a series of critical reflections. The case studies include a range of research foci and methods, from large-scale quantitative secondary data analysis, through interviews and workshops, to ethnographic and participatory methods. A series of shorter reflections drawing on all case studies consider the negotiation of the researcher role, building relationships, the ways knowledge is constructed, the role of place and power, keeping hold of messiness and complexity, ethical practice; and 'slow research'. The principles outlined in this volume are relevant for all research on learning, whenever and wherever it takes place – whether in school or out-of-school.

Case Studies

Chapter 1

Case Study 1 – Out-of-School Activities and the Attainment Gap: A Mixed Methods Exploration of Secondary Data Analysis with Narrative Intersectional Analysis

Karen Laing, Laura Mazzoli Smith and Liz Todd

Abstract

This chapter describes methodologies used in the project ‘Out-of-school activities and the education gap’. The project explored how the out-of-school environment affects children, whether it impacts on primary school attainment and whether it reinforces existing socioeconomic differences. A mixed-methods approach combined three areas of research: statistical analysis of the Millennium Cohort Study (MCS) linked to the National Pupil Database (NPD); a qualitative study through interviews with key stakeholders in 10 schools in London and the North East and the articulation of theories of change for how out-of-school activities may affect attainment. Patterns in how children spend their time, and whether and how this affects attainment, were investigated by analysis of the MCS linked to the NPD. Qualitative research with parents, teachers, pupils and activity providers from schools in London and the North-East afforded an in-depth understanding of drivers and barriers influencing how children spend their time and pathways by which activities may affect children's learning and development. The qualitative research also provided a narrative intersectional analysis of responses in terms of class, gender, ethnicity, religion and disability. Mixing quantitative and qualitative research was made difficult by the volume of data and the time needed to analyse and report each area separately, the different nature of data in the three areas of research and the timing of each phase of data collection. However, meaningful combining of methods occurred at the level of research questions and contributed to a more critical analysis of children's out-of-school activities than had been possible before.

Keywords: Millennium Cohort Study (MCS); National Pupil Database (NPD); theory of change; intersectionality; out-of-school activities; mixed methods

Introduction

The project ‘Out-of-school activities and the education gap’ investigated whether and how out-of-school activities affect primary school children's attainment. The research was carried out in 2015–2017 with a team of researchers at an independent research agency and by the authors at Newcastle University's Centre for Learning and Teaching. It was funded by Nuffield Foundation. We defined extracurricular activity as ‘a learning activity outside normal school hours that children take part in voluntarily’. We looked at both organised activities with adult supervision and unsupervised activities (such as watching TV, caring, homework and reading).

This chapter considers our methods and how we intended to use them to be better placed to understand children's out-of-school lives and the implications of certain aspects of those lives for a range of outcomes. We aimed to combine large-scale secondary data analysis, purposefully collected qualitative data analysis of interviews and theoretical analysis. This chapter looks at the more comprehensive picture of children's out-of-school lives that we were able to present as a result of the research and at the challenges and potential affordances in not fully realising the ambition to combine different areas of data. First, we summarise our methods and discuss the trajectory of how we approached combining them. Next, we briefly present the kinds of analyses we carried out and illustrate these with some of our main findings. Finally, we discuss how far we were actually able to combine the methods and consider some issues in using diverse datasets to approach educational questions about children's out-of-school lives.

Background

The Need for a Mixed-methods Approach

Children's out-of-school time is shaped in a number of ways. They engage in activities, both structured for them by others and that they engage in without adult organisation. The arrangements for caring for children vary, and many young people themselves also have caring responsibilities. These arrangements and responsibilities shape and are shaped by some of their relationships. The engagement of children in activities both at home and outside can provide opportunities to develop other relationships and skills. The ways that children spend their time influence children's social and cultural capital by connecting them to networks of people and knowledge. We know that children differ in their responsibilities and in the activities engaged in. Some of the differences amount to inequalities in what is accessible to them according to a range of factors including the economic capital available to them.

Much was known about the effect of the home learning environment on very young children, but less about the varied nature of out-of-school environments and how these affect older young children. Previous research tended to focus on the out-of-school activities of a wide age range or on older children (Sutton Trust, 2014). Whilst previous research had looked at broad differences in background, there was no fine-grained analysis of any difference across activities of children according to varied ages and backgrounds. Where there has been detailed analysis, it has often been of particular activities or of a relatively small sample of children (Anderson, Wilson, & Fielding, 1988; Becker, Dearden, & Aldridge, 2000). There was a pressing need for larger-scale and more detailed in-depth research that could look at the possible relationship between out-of-school activities and educational attainment. What was needed was a more comprehensive analysis of how children spent their out-of-school time, an analysis of the link between access to out-of-school activities and attainment, particularly for disadvantaged children, and an analysis of how out-of-school activities are accessed by and offered to young people. It became clear to us that both a more large-scale comprehensive analysis and a more fine-grained analysis were needed across children's activities and interests – rather than of a small group of activities – and at a scale that could look at background effects and association with attainment.

We applied to do this research partly because the secondary data had become available: there was another wave of the Millennium Cohort Study (MCS) (11 years), and it was going to be possible for the first time to link the MCS with the National Pupil Database (NPD). It was clear that a more in-depth analysis of the ways that activities were engaged with at the level of the school and the family was needed. A mixed-methods approach to this area was therefore taken, combining large-scale secondary data analysis with case studies that could look in detail at perceptions of and access to activities. The research questions the project aimed to address were the following:

(1) How do children spend their out-of-school time during Key Stages 1 and 2? Are there patterns in how children spend their time in organised activities and at home? How does time use vary by socioeconomic background, gender, ethnicity, current and prior parental employment status, attitudes, school characteristics and region?

(2) Does children's time use in out-of-school hours affect attainment? Which types of activities predict attainment? Are some activities stronger predictors than others? What are the mechanisms underlying this? How does the relationship between activities and attainment vary for different children?

Mixed-methods approach to social science research has a robust literature going back to Denzin's ideas of triangulation (Denzin, 1970). Greene, Kreider, and Mayer (2005) state that

Using methods that gather and represent human phenomena with numbers (such as standardized questionnaires and structured observation protocols), along with methods that gather and represent human phenomena with words (such as open-ended interviews and unstructured observations), are classic instances of mixing data gathering and analysis techniques.

(p. 274)

They argue that mixing methods has the potential to offer a better understanding of social phenomena.

There were three components, both quantitative and qualitative, to our research which our analysis aimed to combine:

• Secondary data analysis, using the MCS to describe what a large number of children do out of school and linking to the NPD to explore associations with educational outcomes;
• School case studies, enabling the analysis of the extracurricular provision of schools, looking at 10 schools in London and 10 in Newcastle and
• Theory building, looking at possible theories for any links between out-of-school activities and educational outcomes.

Secondary data analysis used the MCS and the NPD. This was the first time that the MCS was linked to the NPD to analyse the possible association with attainment. The MCS is a longitudinal birth cohort study that has been tracking 19,000 children born in the United Kingdom during 2000–2001. We carried out secondary analysis of the MCS to identify patterns in how children spend their time and the activities they are involved in. Data on 11,762 UK cohort children were available to us at three time points during their primary school education: at ages 5, 7 and 11. We examined activities that are not only organised (i.e., structured) but also informal, and we consider activity in relation to a number of background factors such as economic disadvantage, class, ethnicity, gender and geographical location. Aspects of the frequencies of some of the activities were recorded. These data were analysed to form a picture of the activities children took part in outside the classroom. Structured activities included breakfast clubs; after school clubs; religious activities; music lessons; sports clubs and childcare. Informal activities included watching TV; reading for pleasure; homework; playing out unsupervised; socialising with friends; playing with parents; doing chores and caring responsibilities. Latent class analysis was used to see whether children could be grouped based on the organised activities they took part in at different time points. This technique divides individuals into discreet, non-overlapping, groups on the basis of their answers to survey questions (or in this case, children's parents' answers). Although there was much we could find out by interrogating this database (for example, many questions including sub-questions about frequency), there were limits as a result of the nature of the data that had been collected. There was also a lot of detail about the activities that were not included in the survey questions, and there were areas of out-of-school activities that were omitted.

In addition to our analysis of the uptake of out-of-school activities by the whole UK sample, we looked at links between English pupils' access to out-of-school activities and their educational attainment. We also looked at whether there were particular effects for those facing economic disadvantage. To do this, the MCS was linked to the NPD and analysed using multiple linear regression and binary logistic regression to consider relationships between out-of-school activities and attainment. We used this to examine the activities of a sub-sample of 6,430 disadvantaged English children (26% of the total). For the purposes of this paper, disadvantage was defined as below 60% of median equivalised family income at two of the three time periods (i.e., poor in two out of 5 year, 7 year and 11 year). These children were from England only since the disadvantage analysis was carried out only on the children linked to the English NPD.

School case studies used qualitative data from interviews with head teachers and teachers, parents, pupils and activity providers in 10 schools in London and the North East. We looked at what was accessed through different schools and at perceived barriers, facilitators and impacts. The schools varied in size, in the proportion of children eligible for free school meals (which varied from 20% to 60%) and in the proportion of children assessed as needing support for special education needs and disability. The interview sample was of course not the same sample as the MCS. Parents interviewed were from a wide diversity of backgrounds, in keeping with the diversity of schools accessed. By starting from schools, and interviewing stakeholders about what was accessed through schools, this meant that there would be some limitations in terms of what could be revealed generally about out-of-school activities. However, we also spoke to a number of providers about a wider group of out-of-school activities.

Stakeholder interviews were subject to two different analyses. The initial analysis looked at the drivers and barriers influencing how children spend their time. This was followed by a thematic analysis looking beyond drivers and barriers. This was a multi-level narrative intersectional analysis looking at responses of all those interviewed in terms of class, gender and ethnicity, religion and disability. The key areas of focus in the qualitative data analysis were patterns of use and function of (or reasons for engagement in) out-of-school activities. Given the in-depth and narrative nature of some of the interview data (particularly with respect to the head teachers), we were also able to locate ‘core stories’ in particular cases of individuals whose responses were comprehensive in exploring out-of-school provision from a range of angles.

It was decided that intersectionality (Crenshaw, 1989) would be a useful way to approach these data because of the many factors that would be working together to influence patterns of engagement in out-of-school activities in diverse and mutually influencing ways. An intersectional approach addresses how related identity categories (e.g., class, ethnicity, race, religion, gender, disability, age, sexuality) overlap to create inequality of access on multiple levels and shape the experiences and values attached to them that we are interrogating. We felt that this was a useful framing of the considerable amount of qualitative data generated in this project. An intersectional approach foregrounds the idea that understanding any social category requires the analysis of difference as well as similarity: one-dimensional categories can perpetuate assumptions that actually contribute to inequality (Museus, 2009). This was an important consideration in ongoing thinking around combining datasets.

Theory building: Drawing on the existing research and literature, we developed theories exploring the anticipated impacts of out-of-school activities on children in our secondary data analysis, linked to the project aims and questions. Theories of change were developed through round-table meetings with out-of-school activity providers and the project academic advisors. For example, based on the literature, we anticipated seeing improved educational outcomes for those children who take part in school-based activities outside of regular school hours. This may be because the activities change their view of school, the teachers see the pupils in a different light or the child's confidence and related aspirations and achievements are developed.

Findings

We found a myriad of ways that children spend their time out of school. We found more robust evidence than in the past of a link between out-of-school activities and attainment, particularly for economically disadvantaged children. Our qualitative data revealed rich and varied intersectional narratives about out-of-school activities. We generated many different theories. An indication of some of our resulting analysis is useful here in order to situate our wider argument in the section following this about how challenging it would be to combine the different datasets or analyses here. The separate reporting of these data is an indication of how difficult it was to combine the data. Findings are reported in a number of brief papers and a working paper (Chanfreau et al., 2015a, 2015b, 2016, 2016a, 2016b).

• Secondary data analysis: Economically disadvantaged pupils had lower take-up of most organised activities than their more affluent peers, except for after-school clubs. However, by age 11, participation in after-school clubs did not vary by economic disadvantage: 31% of both disadvantaged and more affluent children attended after-school clubs at least weekly. In comparison, disadvantaged children were less likely to participate in other activities outside school, including sports activities (61% compared to 78% among more affluent children), extra tuition (18% compared to 24%) and music lessons (7% compared to 29%). The reasons for take-up of after-school clubs included the low cost, convenience associated with them taking place at school and the reassuring familiarity of staff and location. After-school club attendance was associated with positive academic and social outcomes for disadvantaged children in particular. Organised physical activities were associated with higher attainment and better social, emotional and behavioural outcomes at age 11. School staff, parents and pupils identified a wide range of perceived benefits from taking part in after-school clubs that covered academic as well as social and emotional outcomes. Participation ...

Table of contents

• Cover
• Series Title
• Title
• Copyright
• Table of Contents
• List of Tables and Figures
• About the Editors
• About the Contributors
• Acknowledgements
• Introduction
• Case Studies
• Thematic Chapters
• Concluding Thoughts
• References
• Index