DevOps for VMware Administrators

Trevor Roberts, Jr. is a Senior Technical Marketing Manager for VMware. Trevor has the CCIE Data Center certification, and he is a VMware Certified Advanced Professional in the Data Center Design and Administration concentrations. In his spare time, Trevor shares his insights on data center technologies at http://www.VMTrooper. com, via the vBrownBag Professional OpenStack and Professional VMware podcasts, and on Twitter (@VMTrooper). His contributions to the IT community have garnered recognition by his designation as a VMware vExpert, Cisco Data Center Champion, and EMC Elect.   Josh Atwell is a Cloud Architect for SolidFire, focusing on VMware and automation solutions. Over the past 10+ years, he has worked very hard to allow little pieces of code to do his work for him through various automation tools. Josh is a father of two boys with wife Stephanie, and a daughter is on the way in early 2015. Based in the Raleigh, North Carolina, area, he enjoys time with his family, golf, audiobooks, and trying new bourbons. Josh has been highly active in the virtualization community, where he’s been a leader of technology-based user groups such as CIPTUG, VMUG, and UCS Users Group. Josh has worked with others on preparing for their professional development pursuits through the vBrownBag podcast and the Virtual Design Master competition. Josh is also a regular public speaker and contributing author to the Mastering vSphere series. Never known for lacking an opinion, he blogs at vtesseract.com and talks shop on Twitter as @Josh_Atwell.   Egle Sigler (@eglute, anystacker.com) is currently a Principal Architect at Rackspace. She started her career as a software developer, and still has a soft spot for all the people that write, test, and deploy code, because she had a chance to do all of those tasks. Egle dreams about a day when writing, testing, and deploying code will be a seamless and easy process, bug and frustration free for all. Egle believes that knowledge should be shared, and tries to do so by writing this book, giving talks and workshops at conferences, and blogging.   Yvo van Doorn has more than a decade of system administration experience. The first part of his career, he manually built out and configured bare-metal servers. At Classmates, Yvo became a champion of configuration management and virtualization. Before joining Chef, he learned firsthand the power of VMware’s products when he moved a small Seattle technology company’s complete production stack over to its virtualization platform. He’s a strong believer in the culture change that comes with DevOps. When he isn’t busy spreading the gospel of Chef, he’s probably enjoying a hoppy IPA, exploring the great outdoors, or celebrating his Dutch heritage while eating a wheel of gouda and watching Oranje lose the World Cup. Yvo lives with his wife and black lab in Seattle, Washington.

About the Authors xvii

About the Reviewers xviii

Acknowledgments xix

About the Contributing Author xx

Introduction xxii

About This Book xxiii

You the Reader xxiii

What This Book Covers xxiii

Part 1 Introduction to DevOps

1 Introduction to DevOps 1

An Overview of DevOps Principles 1

Implement Systems Thinking 3

  Change the Approach to Team Interactions 3

  Change the Approach to Infrastructure Deployment 5

  Change the Approach to Software Development and Deployment 6

  Collect and Respond to Useful Systems Feedback Often and Adjust Accordingly 7

Furthering Your DevOps Knowledge and Skills 7

Summary 8

References 8

2 DevOps Tools 9

Organizing for Success: Kanban 9

Server Deployment 13

Confi guration Management 14

Continuous Integration 14

Log Analysis 15

Summary 15

References 15

3 Setting Up a DevOps Confi guration Management Test Environment 17

Environment Provisioning with AutoLab 17

Environment Provisioning with Vagrant 18

Creating Images with Packer 23

Managing Source Code 24

   Using Git 24

Summary 31

References 31

Part 2 Puppet

4 Introduction to Puppet 33

Puppet Architecture 33

  Standalone Deployment 34

  Master-Agent Deployment 34

Preparing Your Puppet Test Lab 37

Puppet Resources 38

Puppet Manifests 39

  Conditional Expressions and Variables 43

Puppet Modules 46

  Puppet Forge 48

  Creating Your First Puppet Module 48

  Puppet Module Initialization Manifest (init.pp) 50

  Templates 51

  Using a Puppet Module 54

  Final Step: Version Control Commit 54

Summary 55

Reference 55

5 Systems Management Tasks with Puppet 57

Optimizing the Web Tier with Data Separation 58

Parameters Class (params.pp) 59

  Hiera 63

  Node Classifi cation 67

  Application Tier 68

Database Tier 70

Implementing a Production Recommended Practice 70

Deploying the Application Environment 71

Summary 71

Reference 71

6 VMware vSphere Management with Puppet 73

Puppet’s Cloud Provisioner for VMware vSphere 73

  Preparing VM Templates 73

  Preparing Puppet Master 74

9780133846478_ROBERTS.indb x 3/5/15 4:20 PM

  Contents xi

VMware’s Management Modules 77

  Using the vmware/vcenter Module 77

Summary 83

References 83

Part 3 Chef

7 Introduction to Chef 85

What Is Chef? 85

Core Philosophies Behind Chef 86

Order of Recipe 86

  Idempotence 86

  API-Based Server 87

  The Client Does All the Legwork 87

  Test-Driven Infrastructure 87

  Chef Terminology 87

Recipe 88

  Cookbook 88

  Attributes 88

  Role 88

  Run List 88

  Resource 88

  Environments 88

The Diff erence Between Hosted Chef and Chef Server 89

  Hosted Chef 89

Chef Server 89

Introduction to ChefDK 90

  What Is ChefDK? 90

  Installing ChefDK 90

Using Knife 92

Creating Your First Hello World Chef Recipe 94

Summary 98

8 Systems Management Tasks with Chef 99

Signing Up for Hosted Chef 100

  Setting Up Local Repo with the Starter Kit 102

Community Cookbooks 105

Setting Up System Management 105

  Prep/Setup System Management Task 1: Managing Time 105

  Prep/Setup System Management Task 2: Managing Root Password 108

9780133846478_ROBERTS.indb xi 3/5/15 4:20 PM

  xii Contents

Confi guring Your Virtual Guests 109

  Installing Chef Client 109

Systems Management Tasks 111

  Running Chef Client 113

  Managing the Root Password 115

  Creating Two Environment Files 116

  Uploading Environment Files to your Hosted Chef Organization 117

  Assigning Each Server to an Environment 118

  Modifying Each Server’s Run List to Run the Managedroot Cookbook 119

  Applying Your Changes to Your Nodes 120

  Validating the Enforced Policy 120

Summary 122

References 123

9 VMware vSphere Management with Chef 125

Knife Plugins 126

  Getting Started with knife-vsphere 128

  Confi guring the knife.rb File 128

  Validating the Confi guration 130

  Putting It All Together 130

Chef Provisioning 134

  Chef Provisioning Architecture 134

  Getting Started with Chef Provisioning 135

  Spinning Up Some Nodes 136

Summary 138

Part 4 Ansible

10 Introduction to Ansible 139

Ansible Architecture 139

Preparing your Ansible Test Lab 141

Ansible Groups 142

Ansible Ad Hoc Command Execution 142

  The Ping Module 143

  The Command Module 144

  The User Module 144

  The Setup Module 144

Ansible Playbooks 144

  Conditional Expressions and Variables 146

Ansible Roles 151

  Templates 154

Ansible Galaxy 156

Summary 157

References 157

11 Systems Management Tasks with Ansible 159

Web Server Deployment 159

The Application Tier 160

The Database Tier 162

Role Structure Optimization 164

VMware Resource Management 166

Summary 171

References 171

Part 5 PowerShell 4.0

12 Introduction to PowerShell Desired State Confi guration (DSC) 173

What Is PowerShell DSC? 174

PowerShell DSC Requirements 175

PowerShell DSC Components 175

  Native Cmdlets 175

  Managed Object Format File 176

  Local Confi guration Manager 176

PowerShell DSC Confi gurations 178

PowerShell DSC Modes 180

  Local Push Mode 181

  Remote Push Mode 181

  Pull Mode 182

PowerShell DSC Resources 184

Summary 186

References 187

13 Implementation Strategies with PowerShell DSC 189

Use Cases for PowerShell DSC in VMware Environments 189

Scripted Deployments of VMs with PowerCLI 190

    Incorporating PowerShell DSC in VM Templates 192

Challenges Implementing PowerShell DSC Confi gurations to New VMs 193

  PowerCLI Invoke-VMscript 193

  PowerCLI Copy-VMGuestFile 195

General Lessons Learned 196

Future Use Cases for PowerShell DSC in VMware Environments 197

Summary 198

References 198

Part 6 Application Deployment with Containers

14 Introduction to Application Containers with Docker 199

What Is an Application? 199

  Hidden Complexity 200

  Dependency and Confi guration Confl icts 200

Linux Containers 200

  Control Groups 201

  Namespaces 201

  Container Management 203

Using Docker 203

  Installing Docker 203

  Docker Daemon 204

  Docker Client 204

  Docker Index 205

  Running a Docker Container 205

  Listing Running Containers 206

  Connecting to Running Containers 206

  Building and Distributing Docker Containers 208

  Dockerfi le 209

  Docker Hub 210

  Docker Versus Virtual Machines 211

  Docker Versus Confi guration Management 211

Summary 212

References 212

15 Running Docker Containers at Scale 213

Container Orchestration 213

Kubernetes 214

  Kubernetes Workfl ow 214

Kubernetes Deployment 215

  CoreOS and Kubernetes Cluster Management Utilities 216

  CoreOS Cluster Deployment 217

  etcd Server Confi guration 222

  Network Overlays with Flannel 223

  Kubernetes Cluster Nodes 223

  Kubernetes Service Deployment 225

  Kubernetes Workload Deployment 226

Platform-as-a-Service with Docker 230

Summary 231

References 231

Part 7 DevOps Tool Chain

16 Server Provisioning Using Razor 233

How Razor Works 233

Using Razor 236

  Razor Collections and Actions 238

  Building Razor Collections 245

Using Razor APIs 257

Razor Components 258

  Razor Server 258

  Razor Microkernel 258

  Razor Client 259

Setting Up Razor 259

  PE Razor 259

  Puppet Install 259

  Install from Source 260

  Manual Release Install 260

  Other Services 260

Summary 263

References 263

17 Intro to the ELK: Elasticsearch, Logstash, Kibana 265

Elasticsearch Overview 265

  Getting Started 266

  Understanding the Index 267

  Working with Data 267

  Installing Plugins 271

  Using Clients 274

Logstash Overview 275

  Getting Started 276

  Confi guring Input to Logstash 276

  Applying Filters 278

 

Kibana Overview 280

  Sharing and Saving 285

  Custom Data Views 286

Summary 286

References 287

18 Continuous Integration with Jenkins 289

Continuous Integration Concepts 289

  Continuous Integration or Continuous Deployment? 290

  Test Automation 290

Jenkins Architecture 292

Jenkins Deployment 293

Jenkins Workfl ow 296

  Jenkins Server Confi guration 296

  Jenkins Build Job 298

  Git Hooks 302

  Your First Build 304

Quality Assurance Teams? 306

  Acceptance Testing 306

  Development Team 306

  Build/Test Infrastructure 307

Summary 307

References 307

Part 8 VMware DevOps Practices

19 VMware vRealize Automation in DevOps Environments 309

Emergence of DevOps 309

Stable Agility 310

People, Process, and Conway’s Law 311

vRealize Automation 312

vRealize Application Services 313

Puppet Integration 315

Code Stream 321

Summary 327

References 327

9780133846478, TOC, 3/9/15