Virtualizing and Tuning Large Scale Java Platforms

Emad Benjamin has been in the IT industry for the past 20 years. He graduated with a Bachelor of Electrical Engineering degree from the University of Wollongong. Early in his career, he was a C++ software engineer. Then, in 1997, he took on his first major project using Java and has focused on Java ever since. For the past 8 years, his main concentration has been Java on VMware vSphere. Emad is a featured speaker at VMworld, Spring-One, UberConf, NFJS, and various other Java user groups around the world. Currently, Emad is a principal architect in the Global Center of Excellence focused on VMware virtualization, providing training and evangelism to various corners of the world.

Preface xv

Chapter 1 Introduction to Large-Scale Java Platforms 1

Large-Scale Java Platform Categories 1

Large-Scale Java Platform Trends and Requirements 2

Compute-Resource Consolidation 2

JVM Instance Consolidation 3

Elasticity and Flexibility 3

Performance 4

Large-Scale Java Platform Technical Considerations 4

Theoretical and Practical Limits of Java Platforms 4

NUMA 9

Most Common JVM Size Found in Production Environments 14

Horizontal Scaling Versus Vertical Scaling of JVMs and VMs 15

Summary 20

Chapter 2 Modern Scalable Data Platforms 21

SQLFire Topologies 24

Client/Server Topology 24

Peer-to-Peer Topology 27

Redundancy Zones 28

Global Multisite Topology 28

SQLFire Features 30

Server Groups 32

Partitioning 34

Redundancy 37

Colocation 38

Disk Persistence 39

Transactions 41

Cache Plug-In 46

Listeners 47

Writers 50

Asynchronous Listeners 52

DBSynchronizer 54

SQLF Commands and DDLUtils 57

Active-Active Architectures and Modern Data Platforms 57

Chapter Summary 61

Chapter 3 Tuning Large-Scale Java Platforms 63

GC Tuning Approach 70

Step A: Young Generation Tuning 71

Step B: Old Generation Tuning 76

Step C: Survivor Spaces Tuning 78

Chapter Summary 78

Chapter 4 Designing and Sizing Large-Scale Java Platforms 79

Designing and Sizing a New Environment for a Virtualized Large-Scale Java Platform 79

Step1: Establishing Your Current Production Load Profi le 80

Step 2: Establish a Benchmark 82

Step 3: Size the Production Environment 95

Sizing vFabric SQLFire Java Platforms: Category 2 Workloads 96

Step A: Determine Entity Groups 97

Step B: Determine the Memory Size of the Data Fabric 100

Step C: Establish Building Block VM and JVM Size and How Many vFabric SQLFire Members Are Needed 105

Understanding the Internal Memory Sections of HotSpot JVM 106

Understanding NUMA Implications on Sizing Large VMs and JVMs 108

vFabric SQLFire Sizing Example 112

Chapter Summary 119

Chapter 5 Performance Studies 121

SQLFire Versus RDBMS Performance Study 121

Performance Results 123

Summary of Findings 126

The Olio Workload on tc Server and vSphere Performance Study 127

Looking at the Results 127

SpringTrader Performance Study 131

Application and Data Tier vSphere Confi gurations 133

The SpringTrader Performance Study Results 137

Performance Differences Between ESXi 3, 4.1, and 5 139

CPU Scheduling Enhancements 140

Memory Enhancements 140

vSphere 5 Performance Enhancements 142

Chapter Summary 143

Chapter 6 Best Practices 145

Enterprise Java Applications on vSphere Best Practices (Category 1) 148

VM Sizing and Confi guration Best Practices 148

vCPU for VM Best Practices 149

VM Memory Size Best Practices 150

VM Timekeeping Best Practices 156

Vertical Scalability Best Practices 156

Horizontal Scalability, Clusters, and Pools Best Practices 158

Inter-Tier Confi guration Best Practices 160

High-Level vSphere Best Practices 165

SQLFire Best Practices and SQLFire on vSphere Best Practices (Category 2 JVM Workload Best Practices) 166

SQLFire Best Practices 168

vFabric SQLFire Best Practices on vSphere 173

Category 3 Workloads Best Practices 181

IBM JVM and Oracle jRockit JVMs 181

GC Policy Selection 184

IBM GC Choices 186

Oracle jRockit GC Policies 187

Chapter Summary 187

Chapter 7 Monitoring and Troubleshooting Primer 189

Open a Support-Request Ticket 191

Collecting Metrics from vCenter 191

Troubleshooting Techniques for vSphere with esxtop 195

Java Troubleshooting Primer 198

Troubleshooting Java Memory Problems 202

Troubleshooting Java Thread Contentions 203

Chapter Summary 204

Appendix FAQs 205

Glossary 229

Best Practices

Best Practice 1: Common Distributed Data Platform 24

Best Practice 2: Client/Server Topology 26

Best Practice 3: Peer-to-Peer Multihomed Machines 27

Best Practice 4: Multisite 29

Best Practice 5: Use Server Groups 33

Best Practice 6: Horizontal Partitioning 37

Best Practice 7: Redundancy 38

Best Practice 8: Colocation 38

Best Practice 9: Disk Persistence 40

Best Practice 10: Transactions 45

Best Practice 11: RowLoader 47

Best Practice 12: Listeners 49

Best Practice 13: Writers 51

Best Practice 14: Asynchronous Listeners 53

Best Practice 15: DBSynchronizer 55

Best Practice 16: VM Sizing and VM-to-JVM Ratio Through a Performance Load Test 149

Best Practice 17: VM vCPU CPU Overcommit 149

Best Practice 18: VM vCPU, Do Not Oversubscribe to CPU Cycles That You Don’t Really Need 150

Best Practice 19: VM Memory Sizing 152

Best Practice 20: Set Memory Reservation for VM Memory Needs 154

Best Practice 21: Use of Large Pages 154

Best Practice 22: Use an NTP Source 156

Best Practice 23: Hot Add or Remove CPU/Memory 157

Best Practice 24: Use vSphere Host Clusters 158

Best Practice 25: Use Resource Pools 159

Best Practice 26: Use Affi nity Rules 159

Best Practice 27: Use vSphere-Aware Load Balancers 160

Best Practice 28: Establish Appropriate Thread Ratios That Prevents Bottlenecks (HTTP threads:Java threads:DB Connections Ratio) 160

Best Practice 29: Apache Web Server Sizing 161

Best Practice 30: Load-Balancer Algorithm Choice and VM Symmetry 164

Best Practice 31: vSphere 5.1 165

Best Practice 32: vSphere Networking 165

Best Practice 33: vSphere Storage 166

Best Practice 34: vSphere Host 166

Best Practice 35: JVM Version 168

Best Practice 36: Use Parallel and CMS GC Policy Combination 168

Best Practice 37: Set Initial Heap Equal to Maximum Heap 170

Best Practice 38: Disable Calls to System.gc() 171

Best Practice 39: New Generation Size 171

Best Practice 40: Using 32-Bit Addressing in a 64-Bit JVM 171

Best Practice 41: Stack Size 172

Best Practice 42: Perm Size 172

Best Practice 43: Table Placements in a JVM 172

Best Practice 44: Enable Hyperthreading and Do Not Overcommit CPU 173

Best Practice 45: CPU Cache Sharing 175

Best Practice 46: vFabric SQLFire Member Server, JVM and VM Ratio 175

Best Practice 47: VM Placement 175

Best Practice 48: Set VM Memory Reservation 175

Best Practice 49: vMotion, DRS Cluster, and vFabric SQLFire Server 176

Best Practice 50: VMware HA and vFabric SQLFire 177

Best Practice 51: Guest OS 177

Best Practice 52: Physical NIC 177

Best Practice 53: Virtual NIC 178

Best Practice 54: Troubleshooting SYN Cookies 179

Best Practice 55: Storage 181

 

 

9780133491203 TOC 12/3/2013