Best Damn Server Virtualization Book Period (eBook)

Front Cover 1
The Best Damn Server Virtualization Book Period 2
Copyright Page 4
Contributing Authors 6
Contents 12
Chapter 1: An Introduction to Virtualization 30
Introduction 31
What Is Virtualization? 31
The History of Virtualization 32
The Atlas Computer 32
The M44/44X Project 32
CP/CMS 33
Other Time-Sharing Projects 33
Virtualization Explosion of the 1990s and Early 2000s 34
The Answer: Virtualization Is... 35
Why Virtualize? 36
Decentralization versus Centralization 36
True Tangible Benefits 40
Consolidation 41
Reliability 43
Security 44
How Does Virtualization Work? 45
OS Relationships with the CPU Architecture 45
The Virtual Machine Monitor and Ring-0 Presentation 47
The VMM Role Explored 48
The Popek and Goldberg Requirements 48
The Challenge: VMMs for the x86 Architecture 50
Types of Virtualization 50
Server Virtualization 50
Storage Virtualization 53
Network Virtualization 54
Application Virtualization 54
Common Use Cases for Virtualization 55
Technology Refresh 55
Business Continuity and Disaster Recovery 57
Proof of Concept Deployments 58
Virtual Desktops 58
Rapid Development, Test Lab, and Software Configuration Management 58
Summary 60
Solutions Fast Track 60
Frequently Asked Questions 63
Chapter 2: Virtualization Technologies 66
Hardware Virtualization Software 67
Operating System-Level Virtualization Software 67
Software Comparison Matrix 68
Chapter 3: Introduction to Microsoft Virtual Server 74
Introduction 75
Microsoft Virtual Server 2005 R2 and Dynamics System Initiative 75
What Is Virtualization, and When Should You Use It? 75
Advantages of Virtualization 76
Production Data Centers 76
Test and Development Data Centers 78
Disaster Recovery 78
What Virtualization Tools Are Available? 79
Third-Party Virtualization Tools 80
How Does Virtualization Work? 81
Virtual PC versus Virtual Server 2005 81
Features That Are Found in Both Virtual PC and Virtual Server 2005 R2 82
Differences between Virtual PC and Virtual Server 2005 R2 82
Supported Features in Virtual PC 83
Supported Features in Virtual Server 2005 R2 83
Host Hardware Support 83
Virtual Hardware 83
Remote Management 83
Security 83
Support for Scripting 83
WMI Support 83
Clustering 83
Scenarios for the Use of Virtual PC 84
Help Desk 84
Training 84
Testing 84
Legacy Applications 85
Virtual Server 2005 versus Virtual Server 2005 R2 85
Summary 86
Solutions Fast Track 86
Frequently Asked Questions 88
Chapter 4: Installing and Configuring a Virtual Server 90
Introduction 91
Determining the Physical Size of the Server 91
Physical Server Sizing 91
Evaluating Existing Physical Servers 91
Planning for New Virtual Servers 91
Installing Virtual Server 2005 R2 92
Preinstallation Tasks 93
Installation Tasks 93
Setting Up a Virtual Server Administration Web Site 103
Virtual Server Administration Web Site Installation 104
Installing in a Multi-Server Environment 104
Virtual Machine Remote Console 114
Setting Access Permissions for the Virtual Machine Remote Console 115
Setting Default Location and Search Paths 118
Setting Resource Allocation 121
Summary 125
Solutions Fast Track 125
Frequently Asked Questions 127
Chapter 5: Virtual Machines 128
Introduction 129
Creating the Virtual Server 129
Using the Administration Web Page 131
Virtual Machine Configuration 134
General Properties 135
The Virtual Machine Configuration File 136
Virtual Machine Additions 138
Memory Properties 139
Hard Disks 141
CD/DVD 142
SCSI Adapters 144
Network Adapters 145
Scripts 147
Floppy Disk Drive 149
COM Ports 150
LPT Ports 151
Adding Hardware to the Server 152
Building the Host Server 152
Processors 152
Memory 152
Storage Systems 153
Network Cards 153
COM an LPT Ports 153
USB Ports 154
Installing a Windows OS 154
Loading the Operating System 155
Booting from the Virtual Floppy Disk Drive 157
Starting the Virtual Machine 159
Navigation 164
Control the Virtual Machine 164
Installing the Operating System 164
Installing Virtual Machine Additions 166
Removing Virtual Machine Additions 171
Installing a Non-Windows OS 172
Creating the Virtual Machine 172
Virtual Machine Additions for Linux 177
Summary 192
Solutions Fast Track 193
Frequently Asked Questions 195
Chapter 6: Virtual Networks 198
Introduction 199
Introduction to Virtual Networks 199
Virtual Networks 199
Viewing the Virtual Networks 201
Virtual Network Properties 204
Using the "Internal Network" 211
Using the Loopback Adapter 212
Installing the Loopback Adapter 213
Configuring Host-to-Guest Networking and File Sharing 221
Using the ICS 227
Creating a Virtual Network 237
Binding a Physical Network Adapter to a Virtual Network 240
Changing the Binding of a Virtual Network 241
Changing the Virtual Network for a Virtual Machine 245
Using the Virtual Server Network Services 247
Summary 252
Solutions Fast Track 253
Frequently Asked Questions 255
Chapter 7: Virtual Disks 256
Introduction 257
Removable Virtual Disks 257
CD/DVD Drive 257
Floppy Disk Drive 259
Virtual Hard Disks 265
Dynamically Expanding Virtual Hard Disk 266
Compacting 268
Converting 275
Fixed-Size Virtual Hard Disk 276
Converting 277
Differencing Virtual Hard Disk 278
Chaining 282
Merging 283
Summary 288
Solutions Fast Track 288
Frequently Asked Questions 290
Chapter 8: Introduction to ADS and Virtual Server Migration Tool 292
Automated Deployment Services 293
Rapid Deployment Using ADS 293
What Components Does ADS Use? 294
ADS Controller Service 294
ADS Network Boot Service 295
ADS Image Distribution Service 295
ADS Host Server Requirements 295
ADS Client-Server Requirements 296
ADS Network and Management Requirements 296
Installing ADS 297
Installation Options 297
Installation Process 298
Post-Installation: Configuring ADS 310
Automatically Accepting New Clients 310
Enabling Multicast 314
Administration Agent Installing 316
Adding Hardware Drivers in the Boot OS 321
Editing Using the Sequence Editor 321
Installing Virtual Server Migration Toolkit onto the Virtualization Server 327
Summary 332
Solutions Fast Track 332
Frequently Asked Questions 334
Chapter 9: Managing Virtual Server 336
Introduction 337
The Management Interface 337
Configuring a Central Virtual Server Management Site 338
Using the Virtual Server COM API 348
Using the Virtual Server Programmer's Guide 349
Connecting to the Virtual Server COM Object 355
Accessing a Virtual Server Using Script 357
Listing Virtual Server Properties 358
Setting Virtual Server Properties 360
Creating a Virtual Machine Using Script 361
Creating a Virtual HardDisk 362
Putting It All Together 363
Creating a Virtual Network Using Script 366
Retrieving Guest OS Information Using Script 367
Changing a Virtual Machine State Using Script 370
The Virtual Machine State Model 370
Attaching Scripts to Virtual Server Events 373
Attaching Scripts to Virtual Machine Events 375
Scripts in Action 375
Summary 380
Solutions Fast Track 380
Frequently Asked Questions 382
Chapter 10: Migrating Physical Machines 384
Introduction 385
Getting the Virtualization Environment Ready for Usage 385
Setting Up the Virtualization tools 385
Installing VSMT and ADS Agent on the Virtual Server Host 386
Creating the Virtual Network 389
Capturing the Physical Machine 391
Hardware Inventory 393
Creating the Scripts 396
Validating Hardware 396
Creating Migration Scripts 397
Data Capture 401
Creating the Virtual Machine on the Virtual Server Host 407
Deploying the Virtual Machine on the Host OS 410
Summary 414
Solutions Fast Track 414
Frequently Asked Questions 416
Chapter 11: Troubleshooting 418
Introduction 419
Troubleshooting Virtual Server 2005 R2 419
Troubleshooting Virtual Server Administration Web Site 419
Troubleshooting LsaLogonUser() failed! 419
Troubleshooting Internal Server Error 500 426
Troubleshooting Access Denied Errors 429
Troubleshooting VMRC Server Disabled Errors 429
Troubleshooting Virtual Server Settings 430
Troubleshooting Disappearing Server Settings 430
Troubleshooting Virtual Network Changes 431
Troubleshooting Virtual Machine Performance Issues 434
Disabling TCP Segmentation Offload 435
Don't Use Network Adapter Auto-Configuration 435
Use ISOs instead of CDs Whenever Possible 436
Don't Overallocate Memory 436
Use a Separate Disk Controller for Guest Machines 436
Troubleshooting Automated Deployment Services 436
Troubleshooting PXE 436
Check the DHCP Configuration 437
DHCP Relay Agent 437
Check for Other PXE Servers 438
Check Your Network Drivers 439
Check Your Storage Drivers 439
Check Your BIOS Clock 439
Troubleshooting the ADS Services 439
Check That the ADS Services Are Running 440
Confirm the ADS Controller's IP Address 440
Check the ADS Certificates 440
Troubleshooting the Virtual Server Migration Toolkit 440
Troubleshooting the Virtual Network Setup 440
Troubleshooting Script Creation 441
Troubleshooting ADS Integration 441
Troubleshooting Migration from VMware to Virtual Server 441
Troubleshooting the Migration Process 442
Imaging Problems 442
IDE Disks Cannot Exceed 127 GB 443
Converted SCSI Disks Fail to Boot 443
Summary 447
Solutions Fast Track 447
Frequently Asked Questions 449
Chapter 12: Introducing Xen 450
Introduction 451
What Is Xen? 451
Features of Xen 453
The XenServer Product Family 453
Xen's Virtualization Model Explored 456
Architecture Overview 456
Processor Architecture 457
Paravirtualization with Xen 457
Xen Domains 459
CPU Virtualization 463
Exceptions 464
CPU Scheduling 465
Time 466
Memory Virtualization 467
Memory Allocation 468
Page Tables and Segmentation 470
Virtual Address Translation 472
I/O Virtualization 474
Device I/O Rings 476
Event Channels 477
Virtual I/O Devices and Split Device Drivers 478
Network I/O 479
Block I/O 480
Trusted Platform Module and Other Devices 480
Driver Domains 480
Software and Hardware IOMMUs 481
SWIOTLB 482
Grant Tables 482
The Xenstore 483
Summary 487
Solutions Fast Track 487
Frequently Asked Questions 491
Chapter 13: Deploying Xen: Demystifying the Installation 492
Introduction 493
Determining Which Xen to Choose 493
System Requirements 494
Thinking Before You Start 495
Installing Xen on a Free Linux Distribution 497
Fedora Core 6 497
VirtManager 508
Installing Windows XP 517
Installing the XenServer Product Family 521
What Is XenServer 521
XenServer Requirements 522
Getting and Installing XenServer 522
Installing the Host 523
Client Installation 530
Installing an Initial Virtual Machine on XenServer 534
Other Xen Installation Methods 539
Using the XenSource Binaries and LVM 539
Configuring Xen 542
Getting Xen on Your Network 544
Summary 548
Solutions Fast Track 548
Frequently Asked Questions 550
Chapter 14: The Administrator Console and Other Native Tools 552
Introduction 553
Native Xen Command-Line Tools 554
The xe Command-Line Interface 554
Installing and Cloning XenVMs 555
Starting Up, Shutting Down, Rebooting, Suspending, and Resuming XenVMs 555
Shutting Down and Rebooting XenHosts 555
Query Options for XenHosts 556
XenServer Administrator Console 556
System Requirements for the Administrator Console 556
Installing the Administrator Console 557
Installing the Administrator Console on Windows (XP/2000/2003) 557
Installing the Administrator Console on Linux 564
Using the Administrator Console 564
Working with Hosts 566
Connecting to a XenHost 567
Powering Off/Rebooting a XenHost 567
Deploying and Configuring XenVMs 568
Creating Xen Virtual Machines 568
Cloning XenVMs 569
Additional XenVM Operations 570
Performance Monitoring 571
Summary 572
Solutions Fast Track 572
Frequently Asked Questions 573
Chapter 15: Managing Xen with Third-Party Management Tools 574
Introduction 575
Qlusters openQRM 575
Xen Management with openQRM 575
Overview 576
General Concepts for the Xen/openQRM Mix 577
Plug-ins and Licensing 578
Installing openQRM 581
System Requirements 582
Installing openQRM 3.1 x Server 583
Installing the openQRM Xen Plug-in 587
Managing Xen with openQRM 589
How the Xen Plug-in Works 589
Using openQRM with Xen Integration 590
Provisioning with openQRM-Pro 594
Enomalism 597
Overview of Enomalism 597
Installing Enomalism 598
System Requirements 598
Installation Walkthrough 599
Using Enomalism to Manage Xen 599
Project ConVirt and XenMan 603
Overview of ConVirt 604
Installing ConVirt 604
System Requirements 604
Installation 605
Using ConVirt to Manage Xen 606
The Dashboard 606
Server Pool Operations 607
Server Operations 608
VM Operations 608
The Image Store 610
Summary 612
Solutions Fast Track 612
Frequently Asked Questions 614
Chapter 16: Deploying a Virtual Machine in Xen 616
Introduction 617
Workload Planning and Virtual Machine Placement 617
Memory 617
CPU 617
Network 618
Installing Modified Guests 620
Installing Red Hat Enterprise Linux 4 620
Installing Unmodified Guests 626
Installing Red Hat Linux Enterprise 5 627
Installing Windows Guests 631
Windows Guest Installation 631
Physical-to-Virtual Migrations of Existing Systems 635
P2V Migration 636
Importing and Exporting Existing Virtual Machines 636
Exporting XenVMs 638
Importing XenVMs 639
Summary 642
Solutions Fast Track 642
Frequently Asked Questions 644
Chapter 17: Advanced Xen Concepts 646
Introduction 647
The Virtual Split Devices Model 647
Advanced Storage Concepts 648
High-Performance Solutions for Xen 648
iSCSI Integration with Xen 648
Copy-on-Write 651
DmUserspace 652
UnionFS 652
Advanced Networking Concepts 653
Bridging VLANs 653
Creating Interface Bonds for High Availability and Link Aggregation 654
Routing, Forwarding, and Other Network Tricks 656
Building a Xen Cluster 657
XenVM Migration 664
XenVM Backup and Recovery Solutions 667
Options for Backing Up Your XenVM 667
Making Xen Part of Your Disaster Recovery Plan 667
Full Virtualization in Xen 668
The New Processors with Virtual Extensions (VT-x and AMD-V) 668
Summary 671
Solutions Fast Track 671
Frequently Asked Questions 673
Chapter 18: Scripted Installation 676
Introduction 677
Setting Up the Scripted Installation 677
Creating the Script 677
Remote Network Installation 684
Summary 685
Chapter 19: An Introduction to ESX Native Tools and How to Use Them 686
Introduction 687
Esxtop 687
Esxtop Overview 687
The Virtual Machine World 689
System World 690
The Service Console World 690
Some Other Helpful Esxtop Metrics 690
%USED 690
%Ready 691
%EUSED 691
%MEM 691
vmkfstools 691
Viewing Contents VMFS Partition 691
Import/Export Files 692
Adding a New Virtual Disk, Blank Virtual Disk, and Extending Existing Virtual Disks 692
vmware-cmd 693
vmkusage 695
Summary 697
Chapter 20: Scripting and Programming for the Virtual Infrastructure 698
Introduction 699
VMware Scripting APIs 699
What Are the VMware Scripting APIs? 701
Installing the VMware Scripting APIs 702
Putting the VMware Scripting APIs to Work for You 703
Working with the VmCOM API 703
VmConnectParams 706
VmCollection 707
VmServerCtl 707
VmCtl 709
Managing Guests with User-Defined Variables 714
Working with the VmPerl API 714
VMware::VmPerl::ConnectParams 715
VMware::VmPerl::Server 716
VMware::VmPerl::VM 717
VMware::VmPerl::Question 719
Putting It All Together 720
Example 1: Disconnecting Devices from Every Registered VM 720
Example 2: Simple GUI to List All Virtual Machines 722
Example 3: Test Automation with VMware 725
VMware Virtual Infrastructure SDK 726
What Is the VMware Virtual Infrastructure SDK? 727
The VI SDK Architecture 727
Overview of the VMware Virtual Infrastructure Web Service 729
What Are Web Services? 729
VMware VI SDK Conformance and Web Service Standards 730
Operations Available Using the Virtual Infrastructure SDK 730
Operations for Basic Web Service Client Interaction 730
Operations for Element Management 730
Operations for Virtual Computing 731
Developing with the Virtual Infrastructure SDK 1.1 732
Preparing the Virtual Infrastructure Web Service 732
Working with the VMware WSDL 735
Virtual Infrastructure SDK 1.1 Concepts and Terminology 737
Path Hierarchy 737
Terminology 738
Programming Logic for the SDK 740
Data Models and Datatypes 740
Developing Your Management Application 741
The Connection Process 742
Handling SSL Certificates 743
Obtaining with Object Handles 745
Retrieving Items and Performing Operations 748
Updating Interior Nodes 751
Developing with the Virtual Infrastructure SDK 2.0 752
Features Added to Virtual Infrastructure 2.0 752
Preparing the Virtual Infrastructure 2.0 Web Service 754
Working with the VMware VI SDK 2.0 WSDLs 756
Virtual Infrastructure SDK 2.0 Concepts and Terminology 757
Data and Managed Objects 757
Managed Entity Inventory 757
Host Agent versus VirtualCenter Feature Set 758
Data Models and Data Types 759
Programming Logic for the VI SDK 2.0 762
Developing Your Management Application 763
Managed Object Browser and Other Tools 763
The Connection Process 768
Handling SSL Certificates 770
Retrieving Property Information 771
Other Retrieval Mechanisms 775
Performing Advanced Operations 776
Power Operations 777
Virtual Machine Migration 777
Working with Snapshots 778
Working with Scheduled Tasks 779
Other VMware SDKs 780
VMware Guest SDK 780
VMware CIM SDK 781
Summary 783
Chapter 21: Building a VM 784
Introduction 785
Creation of Virtual Machines Utilizing Command-Line Tools 785
Creation of a Virtual Machine Configuration File 785
Creating Your Virtual Machine Configuration File 787
Creation of a Virtual Machine Disk File 791
Registering Virtual Machines with ESX Server 792
Scripting Creation of Virtual Machines in ESX Shell 793
Scripting Creation of Virtual Machines in Perl Scripts 799
Modifying Scripted VM Creation with Perl 806
Perl Script Components 808
VmPerl Commands 810
Cloning Virtual Machines Utilizing ESX Shell Scripts 811
Cloning Virtual Machines Utilizing VmPerl Scripts 814
Summary 823
Chapter 22: Modifying VMs 824
Introduction 825
The Virtual Machine VMDK File 825
VMDK Components 827
Version=1 827
CID=2af6d34d 827
parentCID=ffffffff 827
file createType="twoGbMaxExtentSparse" 827
The Size in Sectors Value 828
The Disk Data Base Command 828
The Virtual Machine Configuration vmx File 830
vmx File Components 831
config version = "" 831
Scsi0:0 present = "" 831
Scsi0:0 name = "" 831
Scsi0:0 mode = "" 831
scsi0 present = "" 832
scsi0 virtualDev = "" 832
ethernet0 present = "" 832
ethernet0 connectionType = "" 833
ethernet0 devName = "" 833
ethernet0 networkName = "" 833
Ethernet0 addressType = "vpx" 833
Ethernet0 generatedAddress = "" 833
Ethernet0 virtualDev = "vlance" or "vmxnet" or "e1000" 834
Floppy Drives and CD-ROMs for Virtual Machines 834
Graphics Emulation, Unique Identifiers 834
Priority, VMware Tools Settings, and Suspend 835
isolation tools dnd disable = "True" or "False" 836
suspend Directory = "/vmfs/vmhba1:0:83:1" 836
Autostart, Autostop, and Time Sync Options 836
The tools syncTime Option 836
Virtual Machine Conversion from IDE to SCSI 837
ddb adapterType = "buslogic" 837
ddb adapterType = "lsilogic" 838
Scripted Disconnect of IDE Devices 840
Dynamic Creation of Virtual Machines 843
Summary 851
Chapter 23: Instant Disk: How to P2V for Free 852
Introduction 853
What Is a P2V? 853
P2V Techniques 853
VMware P2V Tool 853
Platespin PowerConvert 854
Barts/Ghost 855
The "Big Secret" of P2V 855
Instant Disk Overview 855
The Bad News 856
Prepping the ESX Host: Setting Up FTP on ESX Host 856
Prepping the Source Machine: Install the SCSI Driver 859
Installing the SCSI Driver in Windows 2000/2003 859
Installing the SCSI Driver in Windows NT 867
Continue Prepping the Source Machine: Validate 870
The Linux Rescue CD 870
Booting the Rescue CD 870
At the Command Prompt 876
Finding the Hard Drives and Storage 877
Linux and Hardware 878
Virtual Disk Files on the VMFS 879
Starting the FTP Process 880
Creating a New Virtual Machine and Pointing It to a New VMDK File 881
Windows VMs 881
Post-P2V 882
Summary 883
Chapter 24: Scripting Hot Backups and Recovery for Virtual Machines 884
Introduction 885
Anatomy of a VM Backup 885
Limitations 888
Layered REDO Logs 889
Hot VM Backup Sample Script 892
Choosing the Target for VM Backups 895
NFS 896
Attributes of NFS for VM Backups 896
Pros 896
Cons 896
CIFS 897
Attributes of CIFS for VM Backups 897
Pros 897
Cons 897
FTP 897
Attributes of FTP for VM Backups 897
Pros 898
Cons 898
VMFS 898
Attributes of Copies to VMFS for VM Backups 898
Pros 898
Cons 899
Existing VM Backup Tools 899
vmsnap pl, vmsnap_all, and vmres pl 900
vmbk pl 900
Commercial Options 901
VMX File Backups 902
Incorporating Hot VM Backups into Your Recovery Plan 905
Crash Consistent State 907
Replication 908
Hot VM Backups as Part of the Recovery Plan 908
1st Step: Take an Inventory of Your Virtual Machines 909
2nd Step: Determine the Recovery Point Objective for Each VM 909
3rd Step: Determine the Recovery Time Objective for Each VM 910
4th Step: Apply the Right Backup Job to the Need 910
5th Step: Document Your Results 911
Hybrid Backup Strategy 911
Summary 914
Chapter 25: The Future of Virtualization 916
Introduction 917
The Unofficial Xen Road Map 917
Performance and Scalability 918
NUMA-Aware Architecture 918
Multicore Processors 920
Smart I/O 921
Operating System Support 922
Support in Linux Distributions 923
Xen and Microsoft 923
Other HVM Guests 924
Beyond the x86 CPU Architecture 924
IA-64 Feature Sync with x86 924
Porting to PowerPC 925
Porting to the UltraSPARC Architecture 926
Architecture Enhancements 927
Control Tools 927
Virtual Hard Disk Images and XenFS 928
Virtual Device Enhancements 928
Virtual Infrastructure in Tomorrow's Data Center 929
Technology Trends Driving Improvements in Virtualization 930
Hardware Economies of Scale 930
Multicore and Multithreaded Computing 931
Solutions for Small and Medium-Sized Businesses 933
Integrated Computing 933
Data Center in a Box 934
Large Enterprises 935
Reliability and Availability 935
Security 937
Compliance 940
The Magic Recipe: Other Hardware and Software Virtualization Trends 940
Increasing Density Further with Blade Servers 941
Storage Virtualization 941
Network Virtualization 941
Summary 943
Solutions Fast Track 943
Frequently Asked Questions 945
Index 946