Database Archiving (eBook)

Front Cover 1
Database Archiving How to Keep Lots of Data for a Very Long Time 4
Copyright Page 5
Dedication Page 6
Contents 8
Preface 14
How this Book is Organized 15
Organizations that are Likely to Need Database Archiving 16
Who this Book is for 16
Acknowledgments 18
Part 1: Archiving Basics 20
Chapter 1: Database Archiving Overview 22
1.1 A Definition of Database Archiving 23
1.2 Forms of Data Archiving 25
1.3 The Data Lifeline 31
1.4 Types of Data Objects 34
1.5 Data Retention Requirements Versus Data Archives 35
1.6 The Database Archives and Other Database Types 38
Summary 38
Chapter 2: The Business Case for Database Archiving 40
2.1 Why Database Archiving is a Problem Today 40
2.2 Implications of not Keeping Data 44
2.3 Data Volume Issues 46
2.4 Change Management 49
2.5 Current Database Archiving Practices 50
Summary 51
Chapter 3: Generic Archiving Methodology 52
3.1 The Methodology 52
3.2 Define Motivation for Archiving 54
3.3 Identify Objects to Archive 58
3.4 Determine When to Put Objects in the Archive 61
3.5 Determine How Long to Keep Objects in the Archive 63
3.6 Determine What to do with Discarded Objects 66
3.7 Determine Who Needs Access to Archives and How 66
3.8 Determine the Form of Archive Objects 67
3.9 Determine Where the Archive will be Kept 69
3.10 Determine Operational Processes Needed 76
3.11 Determine Necessary Administrative Processes 78
3.12 Determine Required Change Processes 78
Summary 79
Chapter 4: Components of a Database Archiving System 82
4.1 The Database Archiving Organization 82
4.2 Archive Application Data Gathering 83
4.3 Archive Application Design 84
4.4 Archive Data Extraction 84
4.5 Archive Data Management 85
4.6 Archive Access 85
4.7 Archive Administration 86
Summary 86
Part 2: Establishing a Database Archiving Project 88
Chapter 5: Origins of a Database Archiving Application 90
5.1 Problems that Lead to Database Archiving Solutions 91
5.2 Recognizing Problems 92
5.3 Assignment of Problems for Initial Study 94
5.4 Initial Problem Study Components 94
5.5 Determining the Basic Strategy for the Archiving Application 97
5.6 The Application Strategy Chart 102
Summary 102
Chapter 6: Resources Needed 104
6.1 People 104
6.2 Authority 106
6.3 Education 107
6.4 Repository 107
6.5 Archive Server 108
6.6 Software Tools 108
6.7 Disk Storage 109
Summary 110
Chapter 7: Locating Data 112
7.1 Inventorying Data 112
7.2 Picking the Archivist's Data 118
7.3 Documenting Data Sources 119
Summary 120
Chapter 8: Locating Metadata 122
8.1 Metadata Definitions 122
8.2 Where to Find Metadata 128
8.3 Selecting a Version of the Metadata 132
8.4 Classifying Data 133
8.5 Documenting Metadata 136
8.6 Keeping up with Changes 137
Summary 138
Chapter 9: Data and Metadata Validation 140
9.1 Matching Data to Metadata 140
9.2 Assessing Data Quality 142
9.3 Assessing Metadata Quality 143
9.4 Validating Data Classification 144
9.5 Documenting Validation Activities 144
9.6 Repeating Validation Activities 145
Summary 145
Part 3: Designing Database Archiving Applications 146
Chapter 10: Designing for Archive Independence 148
10.1 Independence from Application Programs 148
10.2 Independence from DBMS 151
10.3 Independence from Systems 152
10.4 Independence from Data Formats 152
Summary 154
Chapter 11: Modeling Archive Data 156
11.1 The Source Data Model 156
11.2 The Target Data Model 161
11.3 Model Representations 166
Summary 167
Chapter 12: Setting Archive Policies 168
12.1 Extract Policies 168
12.2 Archive Storage Policies 173
12.3 Archive Discard Policies 176
12.4 Validation and Approval 178
Summary 178
Chapter 13: Changes to Data Structures and Policies 180
13.1 Archiving System Strategies for Handling Metadata Changes 180
13.2 Metadata Change Categories 182
13.3 Changes to the Archive Data Model 189
13.4 Managing the Metadata Change Process 190
13.5 Changes to Archive Policies 191
13.6 Maintaining an Audit Trail of Changes 195
Summary 196
Part 4: Database Archiving Application Software 198
Chapter 14: The Archive Data Store 200
14.1 Archive Database Choices 201
14.2 Important Features 204
14.3 False Features 212
14.4 How Choices Stack up 214
Summary 214
Chapter 15: The Archive Data Extraction Component 216
15.1 The Archive Extractor Model 217
15.2 Extractor Implementation Approaches 219
15.3 Execution Options 222
15.4 Integrity Considerations 224
15.5 Other Considerations 228
Summary 232
Chapter 16: The Archive Discard Component 234
16.1 Data Structure Considerations 234
16.2 Implementation Form 238
16.3 Integrity Considerations 239
16.4 Operational Considerations 241
16.5 Audit Trail 244
Summary 246
Chapter 17: The Archive Access Component 248
17.1 Direct Programming Access 248
17.2 Access Through Generic Query and Search Tools 250
17.3 Access Through Report Generators and BI Tools 250
17.4 Selective Data Unload 251
17.5 Accessing Original Data 253
17.6 Metadata Services 255
17.7 Other Access Considerations 255
Summary 257
Part 5: Administration of the Database Archive 260
Chapter 18: Ongoing Auditing and Testing 262
18.1 Responsibility for Ongoing Auditing and Testing 263
18.2 Auditing Activities 263
18.3 Testing Activities 267
18.4 Frequency of Audits and Tests 269
Summary 269
Chapter 19: Managing the Archive Over Time 270
19.1 Managing the Archive Database 270
19.2 Managing the Archive Repository 272
19.3 Using Hosted Solutions 274
19.4 Managing Archive Users 275
Summary 278
Chapter 20: Nonoperational Sources of Data 280
20.1 Retired Applications 280
20.2 Data from Mergers and Acquisitions 282
20.3 e-Discovery Applications 283
20.4 Business Intelligence Data 285
20.5 Logs, Audit Trails, and Other Miscellaneous Stuff 286
Summary 287
Final Thoughts 288
Appendix A: Generic Archiving Checklist 290
Appendix B: Goals of a Database Archiving System 294
Appendix C: Job Description of a Database Archive Analyst 296
Position 296
Job Responsibilities 296
Required Qualifications 297
Other Helpful Qualifications and Characteristics 297
Glossary 298
References 302
Index 304