SharePoint Server 2010 Enterprise Content Management

Todd Kitta is currently employed at KnowledgeLake, Inc., a SharePoint ISV specializing in document imaging and Enterprise Content Management on the Microsoft SharePoint platform. Brett Grego is the Director of Engineering at KnowledgeLake, Inc. Chris Caplinger is the CTO as well as one of the founders of KnowledgeLake, Inc. Russ Houberg is a SharePoint Microsoft Certified Master and a senior architect at KnowledgeLake, Inc.

INTRODUCTION xxix PART I: INTRODUCTION TO ENTERPRISE CONTENT MANAGEMENT CHAPTER 1: WHAT IS ENTERPRISE CONTENT MANAGEMENT? 3 Introduction to ECM 4 A Historical Perspective 4 Document Imaging 4 Electronic Documents 6 COLD/Enterprise Report Management 6 Business Process Management/Workfl ow 6 ECM Components 7 Capture 7 Paper 7 Office Documents 8 E-mail 8 Reports 9 Electronic Forms 9 Other Sources 9 Store and Preserve 9 Software 10 Hardware and Media Technologies 10 Cloud 11 Management Components 12 Document Management 12 Web Content Management 12 Business Process Management and Workflow 12 Records Management 12 Collaboration 13 Delivery 13 Search 13 Viewing 14 Transformation 14 Security 16 Summary 16 CHAPTER 2: THE SHAREPOINT 2010 PLATFORM 17 A Brief History of SharePoint 18 SharePoint 2010 18 Capability Categories 19 Sites 19 Composites 19 Insights 20 Communities 20 Content 21 Search 21 SharePoint Concepts 21 Architecture 23 Development Concepts 26 ECM in SharePoint 2010 27 Managed Metadata 27 Ratings 28 The Content Type Hub 28 Search 28 Workfl ow 28 Document Sets 28 Document IDs 29 Content Organizer 29 Records Management 29 Digital Asset Management 29 Web Content Management 29 Summary 30 PART II: PILLARS OF SHAREPOINT ECM CHAPTER 3: DOCUMENT MANAGEMENT 33 What Is Document Management? 34 Microsoft SharePoint As a Document Management System 35 Document Taxonomy 35 Document Libraries 36 The Document Library Programming Model 36 Columns 39 The Column Programming Model 41 Content Types 44 The Content Type Programming Model 45 Managed Metadata 47 Administering Managed Metadata 48 Creating a Global Term Set 49 Using a Term Set in a Column 50 The Managed Metadata Programming Model 50 The Managed Metadata Service 55 Content Type Syndication 56 The Content Type Syndication Programming Model 57 Management of Managed Metadata Service Applications 58 Location-Based Metadata Defaults 60 Confi guring Location-Based Metadata Defaults 60 The Location-Based Metadata Defaults Programming Model 61 Metadata Navigation 62 Confi guring Metadata Navigation 63 Using Metadata Navigation 63 The Managed Metadata Navigation Programming Model 65 The Document ID Service 66 The Document ID Programming Model 68 Document Sets 69 Implementing Document Sets 69 Creating Custom Document Sets 70 Using Document Sets 70 The Document Set Programming Model 71 Document Control 73 Security 73 Managing Users and Groups 76 The Security Programming Model 76 Check-In/Check-Out 79 How to Check Out a Document 79 Programmatically Checking Out a Document 79 Versioning 81 How to Confi gure Versioning 81 Version History 81 Programmatically Interacting with Version History 82 The Audit Trail 83 The Content Organizer 84 Summary 85 CHAPTER 4: WORKFLOW 87 Workfl ow and ECM 87 Windows Workfl ow Foundation 88 WF Concepts 88 Activities 88 Workfl ow Modes 90 Persistence 90 The Role of Workflow in SharePoint 91 Workfl ow Scopes 92 Item 92 Site 93 Workfl ow Phases 93 Association 93 Initiation 93 Execution 93 Authoring and Workflow Types 94 Out-of-the-Box Workflows 94 The Approval Workflow 95 Declarative Workflows 99 Visio 99 SharePoint Designer Workflows 105 Visual Studio Workflows 114 Improvements 115 Creating a Workflow in Visual Studio: An Exercise 116 InfoPath 125 Out-of-the-Box Workflows 125 SharePoint Designer Workflows 125 Visual Studio 126 Pluggable Workflow Services 126 Why You Need Workflow Services 126 Authoring Custom Workflow Services 127 Workflow Event Receivers 130 Summary 131 CHAPTER 5: COLLABORATION 133 ECM and Collaboration 134 SharePoint Is Collaboration 134 Social Tagging 134 Tags 135 How to Create Tags 135 Tag Cloud 137 Notes 137 How to Create Notes 138 Ratings 139 Enabling Ratings for a Document Library or List 140 How to Rate an Item 140 Bookmarklets 141 Registering the Tags and Notes Bookmarklet 142 Creating Tags and Notes Using Bookmarklets 143 Privacy and Security Concerns 143 Tagging Programming Model 144 Working with Tags Programmatically 145 Working with Notes Programmatically 147 Working with Ratings Programmatically 149 My Sites 151 My Profi le 151 My Content 153 My Newsfeed 153 My Sites Architecture 154 Configuring My Sites 154 Configuring My Site Settings in the User Profile Service Application 156 Enabling the Activity Feed Timer Job 157 User Profiles 157 User Profile Policies 158 User Profile Programming Model 159 Working with a User Profile Programmatically 160 User Profile Service Application 165 People 166 Organizations 166 My Site Settings 167 Synchronization 168 Enterprise Wikis 168 Blogs 169 Microsoft Office Integration 170 SharePoint Workspace 170 Outlook Integration 171 Summary 172 CHAPTER 6: SEARCH 173 Introduction 173 Retrieval: The Key to User Adoption 174 The Corpus Profile 176 What Types of Documents Will Be Crawled? 176 Is an IFilter Available for Full-text Crawling All Document Types? 176 How Many of Each Document Type Will Be Crawled? 177 What Is the Average File Size By Document Type? 177 How Often Are Existing Documents Changed? 177 How Much New Content Will Be Added During a Specific Period of Time? 178 Impact of the Corpus Profile 178 Search Solutions 178 SharePoint Server 2010 Enterprise Search 180 Topology Components 180 Confi guration Components 184 The Search Center 189 Calling the Search API 203 FAST Search for SharePoint 2010 203 Functional Overview 203 Index and Query Processing Path 205 Search Architectures for SharePoint ECM 206 Sample Architectures 207 3-Million-Item Corpus 208 10-Million-Item Corpus 208 40-Million-Item Corpus 208 100-Million-Item Corpus 210 500 Million Documents 211 The Impact of Virtualization 211 Tuning Search Performance 211 Health Monitoring 212 Performance Monitoring 212 Improving Crawl Performance 213 Improving Query Performance 213 Summary 214 CHAPTER 7: WEB CONTENT MANAGEMENT 215 WCM Overview 215 Improvements in 2010 216 Authoring 216 AJAX 216 Accessibility 216 Markup Standards 217 Content Query Web Part 217 Cross-browser Support 217 Rich Media 217 Metadata 217 Spectrum of WCM in 2010 218 The SharePoint Server Publishing Infrastructure 218 Templates 218 Features 219 Security 221 Approve Permission Level 221 Manage Hierarchy Permission Level 222 Restricted Read Permission Level 222 Groups 222 Content Types 223 Content Content Types 223 Infrastructural Content Types 224 Site Content 225 The Anatomy of a Page 226 Master Pages 226 Page Layouts 227 An Exercise with Taxonomy and Layouts 227 Metadata 232 Content Query Web Part 233 Web Part Options 233 Query Options 233 Presentation 234 The Content Authoring Process 235 Authoring Web Content 235 Using the Content Organizer 238 Content Deployment 238 Workflow 239 Enterprise Wikis 239 Other Major Considerations 240 Branding 240 Navigation and Search 240 Targeting Global Users 241 Reporting and Analytics 241 Summary 242 CHAPTER 8: RECORDS MANAGEMENT 243 What Is Records Management? 244 Why Records Management Is Important 244 Microsoft SharePoint as a Records Management System 245 Records Management Planning 245 Identifying Roles 245 Analyzing Content 246 Developing a File Plan 247 Designing a Solution 247 Compliance and SharePoint 248 Managing Records 249 Recordization 250 In-Place Records Management 250 Records Center 253 Content Organizer 255 Workflow in Recordization 258 Programming Model for Recordization 259 Information Management Policy 263 Confi guring Information Management Policy 263 Exporting and Importing Policy Settings 266 Programming Model for Information Management Policy 267 Retention 267 Creating Retention Schedules 267 Programmatically Creating Retention Schedules 268 Auditing 270 Configuring Auditing 270 Reporting 271 Audit Reports 271 File Plan Report 272 eDiscovery 272 Summary 273 CHAPTER 9: DIGITAL ASSET MANAGEMENT 275 SharePoint Server 2010 Digital Asset Management Components 276 The Asset Library 276 Digital Asset Columns 276 Digital Asset Content Types 277 Media and Image Web Parts 278 Media Web Part and Field Control 278 Picture Library Slideshow Web Part 280 Image Viewer Web Part and Field Control 280 Content Query Web Part 280 Digital Asset Management Solution Scenarios 280 Marketing and Brand Management 281 Media Development Project 282 Online Training Center 283 Audio or Video Podcasting 284 Media Resource Library 284 Taxonomy Considerations 284 Storage Considerations 285 Managing Content Database Size 285 Remote BLOB Storage 286 Maximum Upload Size 286 Performance Optimization 287 BLOB Caching 287 Bit Rate Throttling 289 Summary 292 CHAPTER 10: DOCUMENT IMAGING 293 What Is Document Imaging? 294 SharePoint as a Document Imaging Platform 295 Setting Up the Scenario 295 Solution Data Flow Diagram 295 Model-View-ViewModel Primer 296 Creating a Simple WPF Capture Application 298 Architecture and Design 299 Implementation 299 Building the MVVM Infrastructure 299 Building the Target Dialog 300 Building the Main Window 301 Deployment 322 Creating a Simple Silverlight Viewer Web Part 322 Architecture and Design 323 Implementation 323 Building the Image Loader 323 Building the Imaging Web Service 324 Building the Imaging HTTP Handler 325 Building the Viewer Web Part 325 Making the Application Accessible from JavaScript 326 Deployment 327 Deploying the Imaging Services 327 Deploying the Viewer Application as a Web Part 327 Setting Up the SharePoint Infrastructure 338 Confi guring SharePoint Search 338 Creating the SharePoint Content Type 339 Creating the SharePoint Document Library 340 Creating the SharePoint Web Part Page 340 Setting Up the SharePoint Web Part Page 341 Customizing the Advanced Search Box Web Part 341 Customizing the Search Core Results Web Part 343 The Solution from End to End 355 Summary 355 CHAPTER 11: ELECTRONIC FORMS WITH INFOPATH 357 Electronic Forms Overview 357 Is It a Form or an Application? 358 InfoPath Overview 358 What s New in 2010 360 More InfoPath Fundamentals 360 Forms Services 360 Deploying Forms 361 Templates and Form Data 362 Rules 364 External Data 368 Custom Code 370 Publishing 371 Determining a Forms Strategy 372 Creating a Custom Form: An Exercise 374 Form Data and Layout 374 Form Rules 376 Form Submission 376 Publishing the Form 379 Summary 380 CHAPTER 12: SCALABLE ECM ARCHITECTURE 381 Storage Architecture, the Key to Performance 381 Performance Pitfalls 382 Too Few Disks in the Array 382 Shared SAN vs. DAS vs. NAS 383 Content Storage Size Factors 384 Database Storage and Capacity Planning 385 SQL Server Supporting Concepts 386 TempDB 390 Log Files 392 Crawl Databases 393 Content Databases 395 Property Databases 396 Service Application Databases 397 Management Databases 400 Prioritizing Disk I/O 400 Index Partition Storage 401 Storage Tuning and Optimization 401 Storage Performance Monitoring 401 Database Data File Management 402 Remote BLOB Storage 403 When to Implement an RBS Solution 405 RBS Provider Options 406 Backup and Restore Considerations 407 SQL Server Licensing Considerations 407 SharePoint 2010 Scalable Topology Design 408 Knowing the Users, the Corpus, and the Processes 408 Farm Size Defi nitions 409 The Case for Additional Web Servers 412 The Case for Additional Application Servers 412 The Case for Additional SQL Servers 412 Scalable Taxonomy Factors 413 Content Organization and Scalable Taxonomy 414 An Exercise in Scalable Taxonomy Design 415 Content Database Size Supported Limits 416 Performance and Resource Throttling 417 Summary 418 PART III: SHAREPOINT ECM SUPPORT CONCEPTS CHAPTER 13: ECM FILE FORMATS 421 It s Alive Your Document, That Is 422 Microsoft Office Formats 422 Microsoft Office Binary 422 Office Open XML 423 Viewing and Editing Microsoft Office Formats with Offi ce Web Apps 425 Word Automation Services 428 Open Document Format 437 Archive Formats 437 TIFF 438 OCR and iFilters 438 Markup 442 Development 442 PDF 442 OCR and iFilters 442 Markup 443 Development 443 Viewing and Editing 443 Living Document Conversion 444 PDF/A 444 Standardization 445 OCR and iFilters 445 Creating, Viewing, and Editing 446 XPS (Open XML Paper Specification) 446 OCR and iFilters 447 Markup 449 Development 449 Summary 450 CHAPTER 14: THE SHAREPOINT ECM ECOSYSTEM 451 The Microsoft Partner Ecosystem 451 Becoming a Partner 452 ISV/Software Competency 452 The SharePoint Ecosystem 453 Technical Community 453 ISV Solutions 454 ABBYY 455 AvePoint 458 GimmalSoft 460 KnowledgeLake 462 Nintex 465 Summary 467 CHAPTER 15: GUIDANCE FOR SUCCESSFUL ECM PROJECTS 469 Migrating to SharePoint 2010 470 Identifying Content for Migration 470 Extracting Content from the Source System 470 File Shares 470 Internally Developed Document Management Solutions 471 Other Legacy Document Management Solutions 472 Preparing Content for Importing 472 Setting the Content Type 472 Metadata Merge 472 Controlling the Import Process 473 General Metadata Cleanup 473 Importing Content into SharePoint 473 Protocols for Importing Content 474 Web Services 474 SharePoint Server Object Model 477 FrontPage Remote Procedure Calls (FPRPC) 479 Protocols for Updating SharePoint Content 480 SharePoint Server Object Model 480 SharePoint Client Object Model 483 Mapping Legacy ECM Features to SharePoint Solutions 486 Document Versions 487 Metadata-based Security 487 Document Protection and Rights Management 488 Annotations and Redaction 488 Search 488 Scanning and Indexing 488 Records Retention and Disposition 489 Workfl ow 489 Avoiding the Pitfalls 489 Capacity Planning 489 Illegal Characters in Metadata 489 Missing Required Data 490 Content Database Transaction Log Growth 490 Managing Upload File Size Restrictions 490 Upgrading a SharePoint ECM Farm to SharePoint 2010 491 Know Your Farm 491 SharePoint Portal Server 2003 and WSS v2.0 491 Microsoft Office SharePoint Server 2007 and WSS v3.0 492 Imaging or Archive-Only Farm with No Customization 492 Collaboration Farm with Customizations 492 Collaboration Farm with Large Imaging or Archive Site Collections 492 Summary 493 INDEX 495