Executing Data Quality Projects

Ten Steps to Quality Data and Trusted Information TM

Danette McGilvray

• Publisher: Morgan Kaufmann
• Year: 2008
• ISBN:
  9780080558394
• Topic: Computer Science
• Subtopic: Entreprise Applications
• Index: Computer Science

Information is currency. Recent studies show that data quality problems are costing businesses billions of dollars each year, with poor data linked to waste and inefficiency, damaged credibility among customers and suppliers, and an organizational inability to make sound decisions. In this important and timely new book, Danette McGilvray presents her "Ten Steps" approach to information quality, a proven method for both understanding and creating information quality in the enterprise. Her trademarked approach—in which she has trained Fortune 500 clients and hundreds of workshop attendees—applies to all types of data and to all types of organizations.* Includes numerous templates, detailed examples, and practical advice for executing every step of the "Ten Steps" approach.* Allows for quick reference with an easy-to-use format highlighting key concepts and definitions, important checkpoints, communication activities, and best practices.* A companion Web site includes links to numerous data quality resources, including many of the planning and information-gathering templates featured in the text, quick summaries of key ideas from the Ten Step methodology, and other tools and information available online.

Chapter 1 Overview

If the state of quality of your company’s products and services

was the same level of quality as the data in your databases,

would your company survive or go out of business?

–Larry English

A corollary: If the state of quality of your company’s data was

the same level of quality as your company’s products and

services, how much more profitable would your company be?

– Mehmet Orun

---

In This Chapter

The Impact of Information and Data Quality

About the Methodology: Concepts and Steps

Approaches to Data Quality in Projects

Engaging Management

---

The Impact of Information and Data Quality

Information quality problems and their impact are all around us: A customer does not receive an order because of incorrect shipping information; products are sold below cost because of wrong discount rates; a manufacturing line is stopped because parts were not ordered—the result of inaccurate inventory information; a well-known U.S. senator is stopped at an airport (twice) because his name is on a government “Do not fly” list; many communities cannot run an election with results that people trust; financial reform has created new legislation such as Sarbanes–Oxley.¹

Information is not simply data, strings of numbers, lists of addresses, or test results stored in a computer. Information is the product of business processes and is continuously used and reused by them. However, it takes human beings to bring information to its real-world context and give it meaning. Every day human beings use information to make decisions, complete transactions, and carry out all the other activities that make a business run. Applications come and applications go, but the information in those applications lives on.

That’s where information quality comes into play. Effective business decisions and actions can only be made when based on high-quality information—the key here being effective. Yes, business decisions are based all the time on poor-quality data, but effective business decisions cannot be made with flawed, incomplete, or misleading data. People need information they can trust to be correct and current if they are to do the work that furthers business goals and objectives.

A firm’s basis for competition … has changed from tangible products to intangible information. A firm’s information represents the firm’s collective knowledge used to produce and deliver products and services to consumers. Quality information is increasingly recognized as the most valuable asset of the firm. Firms are grappling with how to capitalize on information and knowledge. Companies are striving, more often silently, to remedy business impacts rooted in poor quality information and knowledge.

– Kuan-Tsae Huang, Yang W. Lee,
and Richard Y. Wang²

Tom Redman says it well:

The costs of poor quality are enormous. Some costs, such as added expense and lost customers, are relatively easy to spot, if the organization looks. We suggest (based on a small number of careful, but proprietary studies), as a working figure, that these costs are roughly 10 percent of revenue for a typical organization…. This figure does not include other costs, such as bad decisions and low morale, that are harder to measure but even more important.³

What is the cost to a company of the sales rep, publicly announced to have won the top sales award for the year along with the trip to Hawaii, only to have it rescinded a few days later because the sales data were wrong? Does the resulting embarrassment and low morale influence that sales rep’s productivity and therefore sales, or even his decision to stay with the company? What is the cost to the embassy whose name was splashed across the front pages of a major U.S. city’s newspaper when its visa applications containing sensitive personal and business information, such as Social Security numbers and strategic business plans, were found thrown in an open dumpster instead of being properly disposed of? Does the resulting lack of trust in the management of that information influence another company’s decision to do business in that country?

What Is Information Quality?

Information quality is the degree to which information and data⁴ can be a trusted source for any and/or all required uses. Simply put, it is having the right information, at the right time and place, for the right people to use to run the business, serve customers, and achieve company goals. Quality information is also fit for its purpose—the level of quality supports all of its uses.

Definition

Information quality is the degree to which information and data can be a trusted source for any and/or all required uses. It is having the right set of correct information, at the right time, in the right place, for the right people to use to make decisions, to run the business, to serve customers, and to achieve company goals.

Where Do Information Quality Problems Come From?

Information quality problems may be caused by human, process, or system issues. They are not restricted to older or particular types of systems. Although everyone is aware that data cause problems from time to time, it may be difficult to perceive the extent to which these problems affect the business. Some normal business activities are indicative of data quality problems⁵:

• Correction activities
• Rework
• Reprocessing orders
• Handling returns
• Dealing with customer complaints

Many of these activities do not appear to be associated with information quality, when in fact they are. Since processes and functions are distributed across an organization and many people, the cost and scope of data quality problems are often not visible.

Business processes create, update, and delete data in addition to applying information in many ways. Information technology (IT) teams are responsible for the quality of the systems that store and move the data, but they cannot be held completely responsible for the content. Both IT and the business must share in insisting on clearly articulated requirements, strict testing of systems, and the development of quality processes for data management.

The Information Quality Challenge

I believe that two major trends have created an environment where information quality is getting more of the attention it deserves. One is the increasing number of legal and regulatory data quality requirements. The need for and benefits from information quality have always been there and ready for any organization who invests in it. But human nature being what it is, the threat of bad publicity and high fines and the risk of a CEO going to jail have created the motivation to actually do something about data quality.

The second reason is based on the need for business to see information brought together in new ways. Examples include the need to see what top customers are doing across the enterprise through CRM (Customer Relationship Management), to have data available for decision support through business intelligence and data warehousing, to streamline business processes and information through ERP (Enterprise Resource Planning), and to deal with the high rate of mergers and acquisitions, which require the integration of data from different companies.

All these initiatives require data integration—bringing together data from two or more different sources and combining them in such a way that new and better uses can be made of the resulting information. Data that previously fulfilled the needs of one particular functional area in the business are now being combined with data from other functional areas—often with very poor results. We have different business uses for the same information; different platforms, systems, databases, and applications; different types of data (customer, vendor, manufacturing, finance, etc.); different data structures, definitions, and standards; and data, processes, and technology customized to fit the business, geography, or application. These are the challenges of the current environment.

What we need is the ability to share information with our customers and with each other across the company. We need the ability to find what we need, when we need it, and to be able to trust it when we get it. What is required for that to happen? We must consciously manage information as a resource (a source of help) and as an asset (a source drawn on by a company for making profit). We must have information that is real (an accurate reflection of the real world), recent (up to date), and relevant (that our business and customers need and care about).

This book is here to help.

About the Methodology: Concepts and Steps

“Doctor, my left arm hurts!” The doctor puts your arm in a sling, gives you an aspirin, and tells you to go home. But what if you were having a heart attack? You would expect the doctor to diagnose your condition and take emergency measures to save your life. After you were stabilized you would expect the doctor to run tests, get to the root cause of the heart attack, and recommend measures to correct any damage done (if possible) and prevent another attack from occurring. The doctor would have you come in for periodic tests and follow-up to assess your condition and determine if other measures needed to be taken.

This seems like common sense when talking about our health. But when it comes to data and information, how often do we address the immediate business problem, then go for the “easy fix” (the aspirin and sling) and expect that to take care of our problems? No tests or assessments are run to determine the location or magnitude of the problems, no root cause analysis is performed, and no preventive measures are put into place. And then we are surprised when problems appear and reappear!

This book describes a methodology, Ten Steps to Quality Data and Trusted Information, that represents a systematic approach to improving and creating data and information quality. The methodology combines a conceptual framework for understanding information quality and The Ten Steps process, which provides instructions, techniques, and best practices. The methodology is for practical use—put it to work to create and improve the quality of information in your business and to establish continuous improvement through better information management.

Just as with your own health, you can use the methodology presented in this book to prevent data quality “health” problems and to assess and take action if they appear. This book provides processes, activities, and techniques that will improve your company’s information quality health. Think of it as your “wellness” program for data and information.

The Ten Steps Process

The Ten Steps are explicit instructions for planning and executing information quality improvement projects with detailed examples, templates, techniques, and advice. They combine data quality dimensions and business impact techniques to present a picture of the current state of data and information quality in your business. Data quality dimensions are facets of data quality you can use to measure and manage your data and information quality—which can only be improved if they can be measured. You will choose the data quality dimensions to measure and manage that best address your business needs.

Business impact techniques are quantitative and qualitative techniques for assessing the impact of your information quality on the business. Using them answers the questions “What is the impact of the data quality issues” and “Why should I care?” Results from assessing business impact are used to establish the business case for information quality. They are also used to gain support for and help determine the optimal level of investment in it. Following the assessments of quality and/or business impact, root cause analysis is conducted and appropriate actions for preventing and correcting data quality issues are put into place. Communication is critical to the success of any data quality effort, so it too is one of the Ten Steps that takes place throughout the life of every project.

All of the information contained in The Ten Steps is “how-to.” But just as you want a doctor who understands the theories and concepts of medicine so that specific actions can be correctly applied to your medical concerns, you also need to understand information quality basics so that the “how-to” can be properly applied in the many different situations that arise in your company. For that reason, the key concepts are presented first in this book, followed by The Ten Steps process.

The Key Concepts

The key concepts provide the foundation for understanding what information quality is and what is required to achieve it. They include the Framework for Information Quality, the Information Life Cycle, and the Information and Data Quality Improvement Cycle.

The Ten Steps process describes how to implement the key concepts. Just as with your own health, you can use the methodology presented in this book to prevent data quality “health” probl...