Contexts of Learning Mathematics and Science

Lessons Learned from TIMSS

Sarah J. Howie, Tjeerd Plomp

• Publisher: Routledge
• Year: 2006
• Print ISBN: 9780415362252
• eBook ISBN: 9781134232154
• Edition: 1
• Topic: Education
• Subtopic: Education General

This book is the result of research from over fifteen countries, asking which background and environmental factors influence achievement in mathematics and science. This research is based on data from the Third International Mathematics and Science Study (TIMSS), which was conducted under the auspices of the International Association for the Evaluation of Educational Achievement (IEA) in 1995 and 1998.

In many countries researchers have started secondary analysis of the data in search for relationships between contextual factors and achievement. In these analyses two different approaches can be distinguished, which can be characterised by the metaphors of 'fishing' and 'hunting'. In the 'fishing' approach, researchers begin with an open mind, considering all possible context variables as potentially influential. Applying analysis techniques such as regression analysis, Lisrel, PLS, HLM, and MLN, they then identify important factors within their countries or across a number of countries. In the 'hunting' approach, researchers hypothesise certain context variables and trace the effect of these variables on mathematics and/or science achievement.

Part 1: Introduction to TIMSS and to the Book

1: Lessons from Cross-national Research on Context and Achievement: Hunting and Fishing in the TIMSS Landscape

Sarah J. Howie and Tjeerd Plomp

Introduction

This book brings together results of research from across the world that addresses questions on what background and contextual factors are related to students’ achievement in mathematics and sciences. Researchers from 16 countries all utilized data from the Third International Mathematics and Science Studies (TIMSS), which was conducted in 1995 and 1999 under the auspices of the International Association for the Evaluation of Educational Achievement (IEA).

The TIMSS was conducted in 41 countries around the world between 1992 and 1999, with data collection taking place in 1995. The study was designed to assess students’ achievements in mathematics and science in the context of the national curricula, instructional practices and the social environment of students. Testing was carried out in more than 40 countries and in three populations: population 1 being 3rd and 4th grade in most countries; population 2 being 7th and 8th grade in most countries; and population 3 being the final year of secondary school.

TIMSS was repeated for population 2 in 1998 and 1999 in 38 countries in Europe, Africa, North and South America, Asia and Australasia. The repeat study is called TIMSS 1999.

The purpose and the design of the TIMSS studies are presented in Chapter 2. This chapter provides the ‘environment’ of TIMSS as an international study of achievement in mathematics and sciences, as well as the scope and an overview of the book. First, a characterization of the IEA will be given, followed by a brief discussion of functions and goals of international comparative studies of educational achievement, of which TIMSS is an outstanding example. Then the organization of TIMSS will be described, which will be followed by the scope and structure of the book.

International Association for the Evaluation of Educational Achievement – IEA

The IEA is a non-governmental organization with more than 60 member countries that conducts international comparative studies in education. Founded in the early 1960s, the IEA’s mission is to contribute to enhancing the quality of education through its studies. Its international comparative studies have two main purposes: to provide policy-makers and educational practitioners with information about the quality of their education in relation to relevant reference countries; and to assist participating countries in understanding the reasons for observed differences within and between educational systems (Plomp, 1998). To achieve these two purposes, the IEA strives in its studies to make two kinds of comparisons. The first consists of straight international comparisons of the effects of education in terms of scores (or sub-scores) on international tests. The second focuses on the extent to which a country’s intended curriculum (what should be taught in a particular grade) is implemented in schools and is attained by students. The latter kind of comparison focuses mainly on national analyses of a country’s results in an international comparative context. As a consequence, most IEA studies are curriculum-driven.

Over the years, the IEA has conducted studies in major school subjects, such as mathematics, science, written composition, reading comprehension, foreign languages and civics education. The IEA’s most recent studies, in addition to TIMSS, are the Second Civics Education Study, the Progress in International Reading Literacy Study (PIRLS) and the Second International Technology in Education Study (SITES). In addition the Teacher Education and Development Study (TEDS) recently has commenced. With respect to TIMSS, the present IEA policy is to have a four-year cycle for TIMSS under the name ‘Trends in Mathematics and Science Study’ – the most recent TIMSS study was TIMSS 2003 (the results of which were released in December 2004), which will be followed by TIMSS 2007.

More information about the IEA, its studies and the international reports and databases can be found on the IEA web site (http://www.iea.nl). Chapter 2 provides the references of the TIMSS international reports.

International Comparative Achievement Studies

The development of large-scale international comparative studies of educational achievements dates back to the late 1950s and early 1960s when the IEA began its first international mathematics study. Large-scale international comparative studies were made possible by developments in sample survey methodology, group testing techniques, test development and data analysis (Husén and Tuijnman, 1994: 6). The studies involve extensive collaboration, funding and negotiation between participants, organizers and funders, resulting in a long-term commitment of all those involved in a study. These types of studies usually have a variety of purposes, such as to compare levels of national achievement between countries; to identify the major determinants of national achievement, country by country, and to examine to what extent they are the same or different across countries; and to identify factors that affect differences between countries (Postlethwaite, 1999: 12). The functions of these studies have been analysed and described by Kellaghan (1996), Plomp (1998) and Postlethwaite (1999). Plomp et al . (2002) and Howie and Plomp (2005) list these functions as description (mirror) benchmarking, monitoring, enlightenment, understanding and cross-national research.

The studies published in this book illustrate the potential and relevance of the last two functions. Various chapters contribute to the understanding of differences between or within educational systems, and provide information that is helpful in making decisions about the organization of schooling, the deployment of resources, and the practice of teaching (Kellaghan, 1996, in Howie and Plomp, 2005). A number of chapters can be characterized as cross-national research as they study variations between educational systems (‘the world as a laboratory’), which may lead to a better understanding of the factors that contribute to the effectiveness of education.

The Organization of TIMSS

The IEA TIMSS studies, as well as the studies in reading (PIRLS), are coordinated from the TIMSS and PIRLS International Study Center in the Lynch School of Education at Boston College (USA) that is responsible for managing the design and implementation of TIMSS and reporting the results for all countries internationally.

TIMSS is a collaborative effort involving dedication, expertise, and participation from individuals and organizations worldwide. In particular, the project receives important contributions and advice from panels of specialists and experts from participating countries that work directly with the TIMSS and PIRLS International Study Center and IEA. The IEA Secretariat (located in Amsterdam, The Netherlands) provides overall support in coordinating TIMSS with IEA member countries and is directly responsible for fund raising, translation verification and organizing the international Quality Control Monitors. The IEA’s Data Processing Center (in Hamburg, Germany) is responsible for the accuracy and comparability of the international data-files within and across countries. Statistics Canada (located in Ottawa, Canada) is responsible for designing and evaluating the samples in TIMSS, and helping participants to adapt the TIMSS sampling design to local conditions. Educational Testing Service (in Princeton, USA) provides support in scaling the TIMSS achievement data.

The TIMSS National Research Coordinators (NRCs) are responsible for implementing the study in their countries and work with international project staff to ensure that the study is responsive to national concerns, both policy-oriented and practical. NRCs review successive drafts of the frameworks, tests and questionnaires. They also provide considerable input about the feasibility of the design and procedures for survey operations. To ensure the relevance of the data, NRCs review the analyses and report at every step for accuracy and utility. Finally, the NRCs work nationally with the schools, teachers and students whose participation in each country is absolutely vital.

A number of organizations have provided support for this series of studies, including the World Bank, the US National Science Foundation, the US National Center for Education Statistics and the United Nations Development Programme (UNDP). This support made it possible for a number of countries to participate in the study that otherwise would not be able to do so. Examples of such countries represented in this book are South Africa, the Philippines and Bulgaria.

Scope and Structure of the Book

As stated, the research reported in this book addresses questions on what background and contextual factors are related to students’ achievement in mathematics and science.

International research (with the size of the studies varying greatly from case studies to large-scale surveys) shows that there are many factors on school, class and student levels that have positive and negative effects on mathematics and science achievement (see Howie, 2002, for an extensive review of the literature). Most of these factors were included in secondary analyses, as they were included in the TIMSS instruments.

The factors on student level represented in the TIMSS database include: socioeconomic status; books in the home; parental education; parental press; pupils’ attitudes to mathematics; family size; jobs in the home; pupils’ aspirations; peer group attitudes; pupils’ self-concept; self-expectations; enjoyment of mathematics; attitudes toward mathematics; gender; age; and time spent on homework. Some countries included a national option in their TIMSS instruments. For example, in South Africa, where a vast majority of the children are being taught in another language than the language of the home, a test of English language proficiency was included, and in Flanders, Belgium, a test of cognitive ability.

On classroom level, the following factors found in the literature were represented in the TIMSS databases: the learning environment; teachers’ characteristics (including gender); streaming; computers; teachers’ competence; teachers’ confidence; education background; teachers’ qualifications; teachers’ methods; class size; time on task; disruptions in class; calculators; content coverage; and assessment.

Finally, on school level the following factors were included in the TIMSS databases: textbooks; time on task; leadership; decision-making; school size; and location.

This book is divided into five parts, of which Part 1, containing the background chapters, sets the context for TIMSS and for the book as a whole. In the analyses reported in Parts 2–5 two different approaches can be distinguished, which can be characterized by the metaphors of ‘fishing’ and ‘hunting’.

In the ‘fishing’ approach, researchers in principle begin with an open mind, considering all available context variables as possible explanatory variables. Applying analysis techniques such as regression analysis, Lisrel, PLS (partial least square analysis), HLM (hierarchical linear modelling), and MLM (multi-level modelling), they try to identify which factors within their countries, or across a number of countries, explain (to a certain extent) mathematics and/or science achievement. These chapters form Parts 2 and 3, both labelled ‘Background Variables and Achievement’, with Part 2 (Chapters 6–12) focusing on within country analysis and Part 3 (Chapters 13–15) on across country analysis.

In the ‘hunting’ approach, researchers have reasons to choose certain context variables upfront and investigate their relationships with achievement. In Part 4 (Chapters 16–20), the authors discuss a number of curriculum related variables in relation to achievement, whereas in Part 5 (Chapters 21–24) the emphasis is on teaching aspects, classroom processes and/or school management variables and their effect upon achievement in several countries.

All the secondary analyses utilize TIMSS 1995 and/or 1999 data. In terms of the topics of the secondary analyses reflected in the analytical chapters (Chapters 6–24), seven studies involve data only from TIMSS 1995 and 11 studies utilize data only from TIMSS 1999 (see Table 1.1). Just one study analyses both 1995 and 1999 data. While most of the chapters (viz. 10) focus on mathematics achievement only, there are three studies on science achievement specifically. However, six of the chapters include both mathematics and science achievement.

In the following sections, a summary is given of the contents of the various parts of the book, including a brief abstract of each chapter.

Part 1: Introduction to TIMSS and to the Book

This part provides both the context to the TIMSS projects as well as setting the scene for the book. Apart from introducing the TIMSS study as an international comparative achievement study and the scope and structure of the book (Chapter 1 by the editors Sarah J. Howie and Tjeerd Plomp), this part provides the introduction to the TIMSS study (Chapter 2) and to the background and contextual variables included in the study (Chapter 3). To illustrate that investigating background and contextual variables in relation to achievement is not a trivial endeavour, two chapters addressing methodological and technical issues are included: Chapter 3, 2nd part, on how background data can be reported; and Chapter 4 about the reliability of data gathered by questionnaires. The final chapter of this part (Chapter 5) presents a ‘cross chapter’ analysis of the findings reported in this book.

Chapter 2, ‘TIMSS: Purpose and Design’, written by the Co-study Directors of the TIMSS project, Michael O. Martin and Ina V.S. Mullis (USA), provides an overview of the purpose and background of TIMSS, the types of information it provides, and...

Table of contents

• Cover Page
• Contexts of Learning: Classrooms, Schools and Society
• Title Page
• Copyright Page
• List of Figures and Tables
• Notes on Contributors
• Foreword
• Preface
• Part 1: Introduction to TIMSS and to the Book
• Part 2: Background Variables and Achievement: Single Country Studies
• Part 3: Background Variables and Achievement: Multiple Country Studies
• Part 4: Curriculum Related Variables and Achievement
• Part 5: Teaching Aspects/Classroom Processes, School Management Variables and Achievement