Chapter 1 Overview

If the state of quality of your company’s products and services

was the same level of quality as the data in your databases,

would your company survive or go out of business?

–Larry English

A corollary: If the state of quality of your company’s data was

the same level of quality as your company’s products and

services, how much more profitable would your company be?

– Mehmet Orun

The Impact of Information and Data Quality

Information quality problems and their impact are all around us: A customer does not receive an order because of incorrect shipping information; products are sold below cost because of wrong discount rates; a manufacturing line is stopped because parts were not ordered—the result of inaccurate inventory information; a well-known U.S. senator is stopped at an airport (twice) because his name is on a government “Do not fly” list; many communities cannot run an election with results that people trust; financial reform has created new legislation such as Sarbanes–Oxley.1

Information is not simply data, strings of numbers, lists of addresses, or test results stored in a computer. Information is the product of business processes and is continuously used and reused by them. However, it takes human beings to bring information to its real-world context and give it meaning. Every day human beings use information to make decisions, complete transactions, and carry out all the other activities that make a business run. Applications come and applications go, but the information in those applications lives on.

That’s where information quality comes into play. Effective business decisions and actions can only be made when based on high-quality information—the key here being effective. Yes, business decisions are based all the time on poor-quality data, but effective business decisions cannot be made with flawed, incomplete, or misleading data. People need information they can trust to be correct and current if they are to do the work that furthers business goals and objectives.

Tom Redman says it well:

What is the cost to a company of the sales rep, publicly announced to have won the top sales award for the year along with the trip to Hawaii, only to have it rescinded a few days later because the sales data were wrong? Does the resulting embarrassment and low morale influence that sales rep’s productivity and therefore sales, or even his decision to stay with the company? What is the cost to the embassy whose name was splashed across the front pages of a major U.S. city’s newspaper when its visa applications containing sensitive personal and business information, such as Social Security numbers and strategic business plans, were found thrown in an open dumpster instead of being properly disposed of? Does the resulting lack of trust in the management of that information influence another company’s decision to do business in that country?

What Is Information Quality?

Information quality is the degree to which information and data4 can be a trusted source for any and/or all required uses. Simply put, it is having the right information, at the right time and place, for the right people to use to run the business, serve customers, and achieve company goals. Quality information is also fit for its purpose—the level of quality supports all of its uses.

Where Do Information Quality Problems Come From?

Information quality problems may be caused by human, process, or system issues. They are not restricted to older or particular types of systems. Although everyone is aware that data cause problems from time to time, it may be difficult to perceive the extent to which these problems affect the business. Some normal business activities are indicative of data quality problems5:

• Correction activities
• Rework
• Reprocessing orders
• Handling returns
• Dealing with customer complaints

Many of these activities do not appear to be associated with information quality, when in fact they are. Since processes and functions are distributed across an organization and many people, the cost and scope of data quality problems are often not visible.

Business processes create, update, and delete data in addition to applying information in many ways. Information technology (IT) teams are responsible for the quality of the systems that store and move the data, but they cannot be held completely responsible for the content. Both IT and the business must share in insisting on clearly articulated requirements, strict testing of systems, and the development of quality processes for data management.

The Information Quality Challenge

I believe that two major trends have created an environment where information quality is getting more of the attention it deserves. One is the increasing number of legal and regulatory data quality requirements. The need for and benefits from information quality have always been there and ready for any organization who invests in it. But human nature being what it is, the threat of bad publicity and high fines and the risk of a CEO going to jail have created the motivation to actually do something about data quality.

The second reason is based on the need for business to see information brought together in new ways. Examples include the need to see what top customers are doing across the enterprise through CRM (Customer Relationship Management), to have data available for decision support through business intelligence and data warehousing, to streamline business processes and information through ERP (Enterprise Resource Planning), and to deal with the high rate of mergers and acquisitions, which require the integration of data from different companies.

All these initiatives require data integration—bringing together data from two or more different sources and combining them in such a way that new and better uses can be made of the resulting information. Data that previously fulfilled the needs of one particular functional area in the business are now being combined with data from other functional areas—often with very poor results. We have different business uses for the same information; different platforms, systems, databases, and applications; different types of data (customer, vendor, manufacturing, finance, etc.); different data structures, definitions, and standards; and data, processes, and technology customized to fit the business, geography, or application. These are the challenges of the current environment.

What we need is the ability to share information with our customers and with each other across the company. We need the ability to find what we need, when we need it, and to be able to trust it when we get it. What is required for that to happen? We must consciously manage information as a resource (a source of help) and as an asset (a source drawn on by a company for making profit). We must have information that is real (an accurate reflection of the real world), recent (up to date), and relevant (that our business and customers need and care about).

This book is here to help.

About the Methodology: Concepts and Steps

“Doctor, my left arm hurts!” The doctor puts your arm in a sling, gives you an aspirin, and tells you to go home. But what if you were having a heart attack? You would expect the doctor to diagnose your condition and take emergency measures to save your life. After you were stabilized you would expect the doctor to run tests, get to the root cause of the heart attack, and recommend measures to correct any damage done (if possible) and prevent another attack from occurring. The doctor would...