Assessing the Accuracy of Remotely Sensed Data

Principles and Practices, Third Edition

Russell G. Congalton, Kass Green

• Publisher: CRC Press
• Year: 2019
• ISBN:
  9780429627712
• Edition: 3
• Topic: Mathematik
• Subtopic: Wahrscheinlichkeitsrechnung & Statistiken

The past 10 years have brought amazing changes to the technologies used to turn remotely sensed data into maps. As a result, the principles and practices necessary for assessing the accuracy of those maps have also evolved and matured. This third edition of Assessing the Accuracy of Remotely Sensed Data: Principles and Practices is thoroughly updated and includes five new chapters. Now 15 chapters long, this text is the only one of its kind to provide geospatial analysts with the requisite considerations, tools, and theory necessary to conduct successful and efficient map accuracy assessments; and map users with the knowledge to fully understand the assessment process to ensure effective use of maps.

See What's New in the Third Edition:

• 
  
  

• All original chapters have been updated to include new standards, practices, and methodologies.

• 
  
  

• A new chapter on planning accuracy assessments.

• 
  
  

• A new chapter on assessing maps created using object-based technologies.

• 
  
  

• Two case study chapters - one showcasing the assessment of maps created from traditional methods, and one on the assessment of object-based maps.

• 
  
  

• Emphasis on considering and planning for positional accuracy in concert with thematic accuracy.

• 
  
  

• An appendix containing the internationally recognized ASPRS Positional Accuracy Standards.

• 
  
  

• A new final chapter summarizing the key concepts, considerations and lessons learned by the authors in their decades of implementing and evaluating accuracy assessments.

Assessing map accuracy is complex; however, the discussions in this book, together with the many figures, tables, and case studies, clearly present the necessary concepts and considerations for conducting an assessment that is both is practical, statistically reliable, and achievable.

1	Introduction

Why Map?

The earth’s resources are scarce. As we continue to add more people to the earth, the scarcity of resources increases, as does their value. From land use conversion throughout the world, to fragmentation of tropical bird habitat, to polar bear habitat loss in the Arctic, to the droughts in Africa, and wars worldwide, people have significantly affected the resources and ecosystems of the world. The ever-increasing world population and need for all types of resources continue to cause the price of these resources to increase and to intensify conflicts over resource allocation.

As resources become more valuable, the need for timely and accurate information about the type, quantity, and extent of resources multiplies. Allocating and managing the earth’s resources requires accurate knowledge about the distribution of resources across space and time. To efficiently plan emergency response, we need to know the location of roads relative to fire and police stations, hospitals, and emergency shelters. To improve the habitat of endangered species, we need to know what the species habitat requirements are, where that habitat exists, where the animals exist, and how changes to the habitat and surrounding environments will affect species distribution, population, and viability. To plan for future developments, we need to know where people will work, live, shop, and go to school. To grow enough food for an ever-increasing population, we need information about the spatial distribution and yields of our agricultural regions. Because each decision (including the decision to do nothing) impacts (1) the status and location of resources and (2) the relative wealth of individuals and organizations who derive value from the resources, knowing the location of resources and how they interact spatially is critical to effectively managing those resources and ourselves over time.

Why Assess the Accuracy of a Map?

Decisions about resources require maps; and effective decisions require accurate maps or, at least, maps of known accuracy. For centuries, maps have provided important information concerning the distribution of resources across the earth. Maps help us to measure the extent and distribution of resources, analyze resource interactions, and identify suitable locations for specific actions (e.g., development or preservation), plan future events, and monitor change. If our decisions based on map information are to have the expected results, the accuracy of these maps must be known. Otherwise, implementing any decisions based on these maps will result in surprises, and often these surprises may be unacceptable.

For example, suppose that you wish to have a picnic in a forest on the edge of a lake. If you have a map that displays forest, crops, urban, water, and barren land cover types, you can plan the location of your picnic. If you don’t know the accuracy of the map, but the map is 100% accurate, you will be able to travel to your forest lakeside location and, in fact, find yourself in a nice picnic spot. However, if the map is not spatially accurate, you may find that your picnic location falls in the middle of the lake rather than on the shore; and if the map is not labeled correctly (i.e., thematically accurate), you may find yourself in a city next to a fountain or in an agricultural field next to an irrigation ditch. Conversely, if you know the accuracy of the map, you can incorporate the known expectations of accuracy into your planning and create contingency plans in situations when the accuracy is low. This type of knowledge is critical when we move from our lighthearted picnic example to more critical decisions such as endangered species preservation, resource allocation, feeding our growing populations, peace-keeping actions, and emergency response.

There are many reasons for performing an accuracy assessment. Perhaps the simplest reason is curiosity—the desire to know how good a map you have made. In addition to the satisfaction gained from this knowledge, we also need or want to increase the quality of the map information by identifying and correcting the sources of errors. Third, analysts often need to compare various techniques, algorithms, analysts, or interpreters to test which is best. Also, if the information derived from the remotely sensed data is to be used in some decision-making process (i.e., geographic information system [GIS] analysis), then it is critical that some measure of its quality be known. Finally, it is more and more common that some measure of accuracy is included in the contract requirements of many mapping projects. Therefore, valid accuracy is not only useful but may be required.

Accuracy assessment determines the quality of a map created from remotely sensed data. Accuracy assessment can be qualitative or quantitative, expensive or inexpensive, quick or time-consuming, well designed and efficient, or haphazard. The goal of quantitative accuracy assessment is the identification and measurement of map errors so that the map can be as useful as possible to the persons using it to make decisions.

The central purpose of this book is to present the necessary theory and principles for conducting a quantitative accuracy assessment along with the practical considerations of how to effectively and efficiently design and implement such an assessment. Throughout the book, we emphasize that no one single recipe exists for conducting an accuracy assessment. Just as there is no one way to produce a map; there is no one way to assess the accuracy of a map. Instead, this book will teach you to consider every aspect of a mapping project and to design and implement the best possible assessment given the strengths and limitations of each mapping project you conduct, fund, or rely on. This book is not written to be an academic review of every possible idea or method ever published on map accuracy assessment. Instead, it is written for the geospatial analyst who wishes to best conduct a valid and effective assessment of their particular mapping project. As such, the considerations and limitations of such an assessment are emphasized here to best lead the analyst through the process.

Types of Map Accuracy Assessment

There are two types of map accuracy assessment: positional and thematic. Positional accuracy deals with the accuracy of the location of map features and measures how far a spatial feature on a map is from its true or reference location on the ground (Bolstad, 2005). Thematic accuracy deals with the labels or attributes of the features of a map and measures whether the mapped feature labels are different from the true or reference feature label. For example, in the picnic example, the earth’s surface was classified as forest, water, crops, urban, or barren. We are interested in both the accuracy of the location of the features, so that we can locate our picnic spot in a forest on the shore of a lake, and in the thematic accuracy, so that we truly end up in a forest and not in a city, desert, or agricultural field that was erroneously mapped as forest.

The accuracy of any map or spatial data set is a function of both positional accuracy and thematic accuracy, and this book considers both. However, because thematic accuracy is much more complex than positional accuracy, the book devotes considerably more attention to thematic accuracy assessment.

Critical Steps in Accuracy Assessment

As previously stated, there is no one single procedure for conducting either a positional or a thematic accuracy assessment. However, all accuracy assessments include these fundamental steps:

1. Consider the factors involved in the assessment.

2. Design the appropriate sampling approach to collect the reference data.

3. Conduct the sampling.

4. Analyze the data.

5. Report the statistics/results.

Each step must be rigorously planned and implemented. First, the accuracy assessment sampling procedures are designed, and the sample areas on the map are selected. We use sampling because time and funding limitations preclude the assessment of every spatial unit on the map. Next, information is collected from both the map and the reference data for each sample site. Thus, two types of information are collected from each sample:

• Reference accuracy assessment sample data: the position or map class label of the accuracy assessment site, which is derived from data collected that are assumed to be correct

• Map accuracy assessment sample data: the position or map class label of the accuracy assessment site derived from the map or image being assessed

Third, the map and reference information are compared with one another, the results of the comparison are analyzed for statistical significance and for reasonableness, and a report is prepared, which presents the methods and results of the assessment. In summary, effective accuracy assessment requires (1) design and implementation of unbiased sampling procedures, (2) consistent and accurate collection of sample data, and (3) rigorous comparative analysis of the sample map and reference data and reporting of the results.

Because there is no one procedure for designing and implementing an accuracy assessment, there are quite a number of important questions to ask and considerations to think about when conducting a valid assessment. This book addresses the most important ones, including:

1. Questions concerning the design of an accuracy assessment sample approach:

• What are the map classes to be assessed, and how are they distributed across the landscape?

• What is the appropriate sampling unit?

• How many samples should be taken?

• How should the samples be chosen?

2. Questions concerning how the reference data should be collected:

• What should be the source of the reference data?

• How should the reference data be collected?

• When should the reference data be collected?

• How do I ensure consistency and objectivity in my data collection?

3. Questions concerning how the analysis should be conducted:

• What are the different analysis techniques for continuous versus discontinuous map data?

• What is an error matrix, and how should it be used?

• What are the statistical properties associated with the error matrix, and what analysis techniques are applicable?

• What is fuzzy accuracy, and how can you conduct a fuzzy accuracy assessment?

• What is object-based accuracy, and how can you conduct an object-based accuracy assessment?

• How is an accuracy assessment conducted on change detection maps?

• How is an accuracy assessment conducted on maps created from multiple layers of data?

Organization of the Book

The organization of this book takes you through each of these fundamental accuracy assessment steps as follows:

• Chapter 2 begins with a review of the history and basic assumptions of map making and accuracy assessment.

• Chapter 3 is a new chapter in this edition and provides the reader with frameworks for planning out accuracy assessments. An important component of any good assessment is proper planning from the beginning of the mapping project. This chapter aids the reader in getting started on making an effective plan by identifying the components of the assessment necessary for their project. It is expected that this chapter will be reviewed by the analyst prior to beginning each and every accuracy assessment.

• Chapter 4 is a thoroughly revised presentation on positional accuracy assessment including a review of past standards and concluding with the latest and most effective ASPRS Positional Accuracy Standards for Digital Geospatial Data (ASPRS, 2014). This complete document is included as an appendix.

• Chapters 5 and 6 provide some of the basic methods and considerations for thematic map accuracy assessment, including the introduction of the error matrix and a thorough review of sample design considerations.

• Chapter 7 is devoted to factors that must be taken into account during the collection of reference data. The collection of suffic...