Developing Tests and Questionnaires for a National Assessment of Educational Achievement

Prue Anderson, George Morgan

• Publisher: World Bank Publications
• Year: 2008
• eBook ISBN: 9780821374986
• Edition: 0
• Topic: Education
• Subtopic: Evaluation & Assessment in Education

'Developing Tests and Questionnaires for a National Assessment of Educational Achievement' is the second in the 'National Assessments of Educational Achievement' series. It is designed to help build capacity in carrying out technically adequate assessments of national levels of student achievement. It introduces readers to the activities involved in the development of achievement tests, and includes developing an assessment framework, writing multiple choice and constructed response type items, pretesting, producing test booklets, and handscoring items. A section on questionnaire construction features designing questionnaires, writing questions, coding responses, and linking questionnaire and test score data. The final section covers the development of a test administration manual, selecting test administrators, and contacting sampled schools. A companion CD contains examples of released items from national and international tests, sample questionnaires, and administrative manuals. The book should serve as a useful resource for those interested in educational assessment and evaluation, especially for personnel engaged in national assessments of student achievement. 'National Assessments of Educational Achievement' Effective assessment of the performance of educational systems is a key component in developing policies to optimize the development of human capital around the world. The five books in the National Assessments of Educational Achievement series introduce key concepts in national assessments of student achievement levels, from policy issues to address when designing and carrying out assessments through test development, questionnaire design, sampling, organizing and carrying out data collection, data cleaning, statistical analysis, report writing, and using results to improve educational quality.

Table of contents

• CONTENTS
• PREFACE
• ABOUT THE AUTHORS AND EDITORS
• ACKNOWLEDGMENTS
• ABBREVIATIONS
• PART 1 CONSTRUCTING ACHIEVEMENT TESTS
• PART 2 CONSTRUCTING QUESTIONNAIRES
• PART 3 DESIGNING A MANUAL FOR TEST ADMINISTRATION
• APPENDIX A GLOSSARY
• APPENDIX B FURTHER READING
• APPENDIX C EXAMPLES OF TEST AND QUESTIONNAIRE ITEMS AND ADMINISTRATION MANUALS ON CD
• INDEX
• Box 2.1 Papua New Guinea Mathematics Curriculum
• Box 2.2 New Zealand English Curriculum
• Box 2.3 Examples of Multiple-Choice Items
• Box 2.4 Example of a Closed Constructed-Response Item
• Box 2.5 Examples of Open-Ended Short-Response Items
• Box 2.6 Example of an Essay Prompt
• Box 3.1 Example of Irrelevant Stimulus Material
• Box 3.2 Example of an Item with Inaccurate or Misleading Information
• Box 3.3 Example of a Multiple-Choice Item
• Box 3.4 Punctuation in Complete Sentences
• Box 3.5 Punctuation in a List
• Box 3.6 Minimizing Reading
• Box 3.7 Item with a Negative Stem
• Box 3.8 Poorly Paired Distractors
• Box 3.9 Dealing with Pairs in Distractors
• Box 3.10 Confusing Open-Ended Item with Unclear Directions
• Box 3.11 Good Example of a Closed Constructed-Response Item
• Box 3.12 Item with Partial Credit
• Box 3.13 Example of an Open-Ended Response Item with Scoring Guide
• Box 3.14 Example of a Closed-Constructed Item with Scoring Guide
• Box 3.15 Using Pictures to Reduce Words
• Box 3.16 Keeping Images Simple
• Box 3.17 Labeling Graphs Clearly
• Box 3.18 Labeling Maps Clearly
• Box 3.19 Leaving Space in the Stimulus Material
• Box 3.20 Example of Style Sheet for Item Writers
• Box 4.1 Example of an Item in Multiple-Choice and Open-Ended Format
• Box 4.2 Example of a Data Entry Sheet for a Pretest
• Box 6.1 Example of a Test Cover Page
• Box 8.1 Attitudes and Values Questionnaire Blueprint
• Box 9.1 Poor Alignment of Boxes and Response Categories
• Box 9.2 Better Alignment of Boxes and Response Categories
• Box 10.1 Grayscale Coding Example
• Box 10.2 Example Treating Items as Separate Categories for Data Entry
• Box 12.1 Administration Manual Instructions
• Box 12.2 Information for Teachers and Principals
• Box 12.3 Administration of Practice Items
• Box 13.1 Administration Checklist: An Example from the Philippines
• Figure 1.1 National Assessment Organizational Chart
• Figure 1.2 Overview of National Assessment Activities
• Figure 4.1 Example of Circular Linking of Items
• Figure 4.2 Model for Vertically Linking Items
• Figure C.1 Guide to CD Materials on Tests, Questionnaires, and Manuals
• Table 1.1 National Assessment Stages in Test Development and Questionnaire Design
• Table 2.1 Blueprint for a Middle Primary Mathematics Test
• Table 2.2 TIMSS Mathematics Blueprint, Grades 3 and 4
• Table 2.3 Papua New Guinea Mathematics Content Blueprint
• Table 2.4 Advantages and Limitations of Item Formats
• Table 2.5 Papua New Guinea Mathematics Test-Item Formats
• Table 4.1 Link Items in Two Reading Units
• Table 4.2 Part of a Spreadsheet for Tracking Items across Forms
• Table 5.1 Example of Output from Analysis of a Multiple-Choice Item
• Table 5.2 Example of Output from Analysis of an Open-Ended Partial-Credit Item
• Table 8.1 Components of Questionnaire Development
• Table 8.2 Functions of Reading in an International Study: Weights Used to Create Two New Variables, “Reading for a Utilitarian Purpose” and “Reading for Enjoyment”