Master Data Management (eBook)

Front Cover 1
Master Data Management 6
Copyright Page 7
Contents 8
Preface 18
About the Approach Described in This Book 19
Overview of the Book 21
More about MDM and Contact Information 21
Acknowledgments 24
About the Author 26
Chapter 1: Master Data and Master Data Management 28
1.1 Driving the Need for Master Data 28
1.2 Origins of Master Data 30
1.2.1 Example: Customer Data 31
1.3 What Is Master Data? 32
1.4 What Is Master Data Management? 35
1.5 Benefits of Master Data Management 37
1.6 Alphabet Soup: What about CRM/SCM/ERP/BI (and Others)? 39
1.7 Organizational Challenges and Master Data Management 42
1.8 MDM and Data Quality 44
1.9 Technology and Master Data Management 45
1.10 Overview of the Book 45
1.11 Summary 47
Chapter 2: Coordination: Stakeholders, Requirements, and Planning 50
2.1 Introduction 50
2.2 Communicating Business Value 51
2.2.1 Improving Data Quality 52
2.2.2 Reducing the Need for Cross-System Reconciliation 52
2.2.3 Reducing Operational Complexity 52
2.2.4 Simplifying Design and Implementation 53
2.2.5 Easing Integration 54
2.3 Stakeholders 54
2.3.1 Senior Management 54
2.3.1 Business Clients 55
2.3.3 Application Owners 55
2.3.4 Information Architects 56
2.3.5 Data Governance and Data Quality 56
2.3.6 Metadata Analysts 57
2.3.7 System Developers 57
2.3.8 Operations Staff 57
2.4 Developing a Project Charter 58
2.5 Participant Coordination and Knowing Where to Begin 59
2.5.1 Processes and Procedures for Collaboration 60
2.5.2 RACI Matrix 60
2.5.3 Modeling the Business 61
2.5.4 Consensus Driven through Metadata 62
2.5.5 Data Governance 63
2.6 Establishing Feasibility through Data Requirements 63
2.6.1 Identifying the Business Context 64
2.6.2 Conduct Stakeholder Interviews 65
2.6.3 Synthesize Requirements 66
2.6.4 Establishing Feasibility and Next Steps 68
2.7 Summary 68
Chapter 3: MDM Components and the Maturity Model 70
3.1 Introduction 70
3.2 MDM Basics 71
3.2.1 Architecture 72
3.2.2 Master Data Model 72
3.2.3 MDM System Architecture 73
3.2.4 MDM Service Layer Architecture 73
3.3 Manifesting Information Oversight with Governance 74
3.3.1 Standardized Definitions 74
3.3.2 Consolidated Metadata Management 75
3.3.3 Data Quality 76
3.3.4 Data Stewardship 76
3.4 Operations Management 76
3.4.1 Identity Management 77
3.4.2 Hierarchy Management and Data Lineage 77
3.4.3 Migration Management 78
3.4.4 Administration/Configuration 78
3.5 Identification and Consolidation 78
3.5.1 Identity Search and Resolution 79
3.5.2 Record Linkage 79
3.5.3 Merging and Consolidation 79
3.6 Integration 80
3.6.1 Application Integration with Master Data 80
3.6.2 MDM Component Service Layer 80
3.7 Business Process Management 81
3.7.1 Business Process Integration 81
3.7.2 Business Rules 82
3.7.3 MDM Business Component Layer 82
3.8 MDM Maturity Model 83
3.8.1 Initial 83
3.8.2 Reactive 83
3.8.3 Managed 86
3.8.4 Proactive 87
3.8.5 Strategic Performance 89
3.9 Developing an Implementation Road Map 90
3.10 Summary 92
Chapter 4: Data Governance for Master Data Management 94
4.1 Introduction 94
4.2 What Is Data Governance? 95
4.3 Setting the Stage: Aligning Information Objectives with the Business Strategy 96
4.3.1 Clarifying the Information Architecture 97
4.3.2 Mapping Information Functions to Business Objectives 98
4.3.3 Instituting a Process Framework for Information Policy 98
4.4 Data Quality and Data Governance 99
4.5 Areas of Risk 99
4.5.1 Business and Financial 99
4.5.2 Reporting 100
4.5.3 Entity Knowledge 100
4.5.4 Protection 101
4.5.5 Limitation of Use 101
4.6 Risks of Master Data Management 101
4.6.1 Establishing Consensus for Coordination and Collaboration 101
4.6.2 Data Ownership 102
4.6.3 Semantics: Form, Function, and Meaning 103
4.7 Managing Risk through Measured Conformance to Information Policies 104
4.8 Key Data Entities 105
4.9 Critical Data Elements 105
4.10 Defining Information Policies 106
4.11 Metrics and Measurement 107
4.12 Monitoring and Evaluation 108
4.13 Framework for Responsibility and Accountability 109
4.14 Data Governance Director 110
4.15 Data Governance Oversight Board 111
4.16 Data Coordination Council 111
4.17 Data Stewardship 112
4.18 Summary 113
Chapter 5: Data Quality and MDM 114
5.1 Introduction 114
5.2 Distribution, Diffusion, and Metadata 115
5.3 Dimensions of Data Quality 116
5.3.1 Uniqueness 117
5.3.2 Accuracy 117
5.3.3 Consistency 117
5.3.4 Completeness 118
5.3.5 Timeliness 119
5.3.6 Currency 119
5.3.7 Format Compliance 119
5.3.8 Referential Integrity 120
5.4 Employing Data Quality and Data Integration Tools 120
5.5 Assessment: Data Profiling 121
5.5.1 Profiling for Metadata Resolution 121
5.5.2 Profiling for Data Quality Assessment 123
5.5.3 Profiling as Part of Migration 123
5.6 Data Cleansing 124
5.7 Data Controls 126
5.7.1 Data and Process Controls 127
5.7.2 Data Quality Control versus Data Validation 127
5.8 MDM and Data Quality Service Level Agreements 128
5.8.1 Data Controls, Downstream Trust, and the Control Framework 128
5.9 Influence of Data Profiling and Quality on MDM (and Vice Versa) 129
5.10 Summary 130
Chapter:6 Metadata Management for MDM 132
6.1 Introduction 132
6.2 Business Definitions 135
6.2.1 Concepts 136
6.2.2 Business Terms 136
6.2.3 Definitions 137
6.2.4 Semantics 137
6.3 Reference Metadata 138
6.3.1 Conceptual Domains 138
6.3.2 Value Domains 139
6.3.3 Reference Tables 140
6.3.4 Mappings 141
6.4 Data Elements 142
6.4.1 Critical Data Elements 143
6.4.2 Data Element Definition 143
6.4.3 Data Formats 144
6.4.4 Aliases/Synonyms 144
6.5 Information Architecture 145
6.5.1 Master Data Object Class Types 145
6.5.2 Master Entity Models 146
6.5.3 Master Object Directory 147
6.5.4 Relational Tables 147
6.6 Metadata to Support Data Governance 147
6.6.1 Information Usage 147
6.6.2 Information Quality 148
6.6.3 Data Quality SLAs 148
6.6.4 Access Control 149
6.7 Services Metadata 149
6.7.1 Service Directory 150
6.7.2 Service Users 150
6.7.3 Interfaces 150
6.8 Business Metadata 151
6.8.1 Business Policies 152
6.8.2 Information Policies 153
6.8.3 Business Rules 153
6.9 Summary 153
Chapter 7: Identifying Master Metadata and Master Data 156
7.1 Introduction 156
7.2 Characteristics of Master Data 158
7.2.1 Categorization and Hierarchies 158
7.2.2 Top-Down Approach: Business Process Models 160
7.2.3 Bottom-Up Approach: Data Asset Evaluation 161
7.3 Identifying and Centralizing Semantic Metadata 162
7.3.1 Example 162
7.3.2 Analysis for Integration 164
7.3.3 Collecting and Analyzing Master Metadata 164
7.3.4 Resolving Similarity in Structure 165
7.4 Unifying Data Object Semantics 166
7.5 Identifying and Qualifying Master Data 167
7.5.1 Qualifying Master Data Types 167
7.5.2 The Fractal Nature of Metadata Profiling 168
7.5.3 Standardizing the Representation 169
7.6 Summary 169
Chapter 8: Data Modeling for MDM 170
8.1 Introduction 170
8.2 Aspects of the Master Repository 171
8.2.1 Characteristics of Identifying Attributes 171
8.2.2 Minimal Master Registry 171
8.2.3 Determining the Attributes Called “Identifying Attributes” 172
8.3 Information Sharing and Exchange 173
8.3.1 Master Data Sharing Network 173
8.3.2 Driving Assumptions 173
8.3.3 Two Models: Persistence and Exchange 176
8.4 Standardized Exchange and Consolidation Models 176
8.4.1 Exchange Model 177
8.4.2 Using Metadata to Manage Type Conversion 178
8.4.3 Caveat: Type Downcasting 179
8.5 Consolidation Model 179
8.6 Persistent Master Entity Models 180
8.6.1 Supporting the Data Life Cycle 180
8.6.2 Universal Modeling Approach 181
8.6.3 Data Life Cycle 182
8.7 Master Relational Model 183
8.7.1 Process Drives Relationships 183
8.7.2 Documenting and Verifying Relationships 183
8.7.3 Expanding the Model 184
8.8 Summary 184
Chapter 9: MDM Paradigms and Architectures 186
9.1 Introduction 186
9.2 MDM Usage Scenarios 187
9.2.1 Reference Information Management 187
9.2.2 Operational Usage 189
9.2.3 Analytical Usage 191
9.3 MDM Architectural Paradigms 192
9.3.1 Virtual/Registry 193
9.3.2 Transaction Hub 195
9.3.3 Hybrid/Centralized Master 196
9.4 Implementation Spectrum 198
9.5 Applications Impacts and Architecture Selection 199
9.5.1 Number of Master Attributes 200
9.5.2 Consolidation 201
9.5.3 Synchronization 201
9.5.4 Access 201
9.5.5 Service Complexity 202
9.5.6 Performance 202
9.6 Summary 203
Chapter 10: Data Consolidation and Integration 204
10.1 Introduction 204
10.2 Information Sharing 205
10.2.1 Extraction and Consolidation 205
10.2.2 Standardization and Publication Services 206
10.2.3 Data Federation 206
10.2.4 Data Propagation 207
10.3 Identifying Information 208
10.3.1 Indexing Identifying Values 208
10.3.2 The Challenge of Variation 209
10.4 Consolidation Techniques for Identity Resolution 210
10.4.1 Identity Resolution 211
10.4.2 Parsing and Standardization 212
10.4.3 Data Transformation 213
10.4.4 Normalization 213
10.4.5 Matching/Linkage 214
10.4.6 Approaches to Approximate Matching 215
10.4.7 The Birthday Paradox versus the Curse of Dimensionality 216
10.5 Classification 217
10.5.1 Need for Classification 218
10.5.2 Value of Content and Emerging Techniques 218
10.6 Consolidation 219
10.6.1 Similarity Thresholds 220
10.6.2 Survivorship 220
10.6.3 Integration Errors 222
10.6.4 Batch versus Inline 223
10.6.5 History and Lineage 223
10.7 Additional Considerations 224
10.7.1 Data Ownership and Rights of Consolidation 224
10.7.2 Access Rights and Usage Limitations 225
10.7.3 Segregation Instead of Consolidation 226
10.8 Summary 226
Chapter 11: Master Data Synchronization 228
11.1 Introduction 228
11.2 Aspects of Availability and Their Implications 229
11.3 Transactions, Data Dependencies, and the Need for Synchrony 230
11.3.1 Data Dependency 231
11.3.2 Business Process Considerations 232
11.3.3 Serializing Transactions 233
11.4 Synchronization 234
11.4.1 Application Infrastructure Synchronization Requirements 235
11.5 Conceptual Data Sharing Models 236
11.5.1 Registry Data Sharing 236
11.5.2 Repository Data Sharing 237
11.5.3 Hybrids and Federated Repositories 238
11.5.4 MDM, the Cache Model, and Coherence 239
11.6.1 Incremental Adoption 242
11.6.1 Incorporating and Synchronizing New Data Sources 242
11.6.2 Application Adoption 243
11.7 Summary 243
Chapter 12: MDM and the Functional Services Layer 244
12.1 Collecting and Using Master Data 245
12.1.1 Insufficiency of ETL 245
12.1.2 Replication of Functionality 246
12.1.3 Adjusting Application Dependencies 246
12.1.4 Need for Architectural Maturation 246
12.1.5 Similarity of Functionality 246
12.2 Concepts of the Services-Based Approach 247
12.3 Identifying Master Data Services 249
12.3.1 Master Data Object Life Cycle 249
12.3.2 MDM Service Components 251
12.3.3 More on the Banking Example 251
12.3.4 Identifying Capabilities 252
12.4 Transitioning to MDM 254
12.4.1 Transition via Wrappers 255
12.4.2 Maturation via Services 255
12.5 Supporting Application Services 257
12.5.1 Master Data Services 257
12.5.2 Life Cycle Services 258
12.5.3 Access Control 259
12.5.4 Integration 259
12.5.5 Consolidation 260
12.5.6 Workflow/Rules 260
12.6 Summary 261
Chapter 13: Management Guidance for MDM 264
13.1 Establishing a Business Justification for Master Data Integration and Management 265
13.2 Developing an MDM Road Map and Rollout Plan 267
13.2.1 MDM Road Map 267
13.2.2 Rollout Plan 268
13.3 Roles and Responsibilities 271
13.4 Project Planning 272
13.5 Business Process Models and Usage Scenarios 272
13.6 Identifying Initial Data Sets for Master Integration 273
13.7 Data Governance 273
13.8 Metadata 274
13.9 Master Object Analysis 275
13.10 Master Object Modeling 276
13.11 Data Quality Management 276
13.12 Data Extraction, Sharing, Consolidation, and Population 277
13.13 MDM Architecture 279
13.14 Master Data Services 280
13.15 Transition Plan 282
13.16 Ongoing Maintenance 283
13.17 Summary: Excelsior! 284
Bibliography and Suggested Reading 286
Bibliography 286
Suggested Reading 286
Index 288