Practical Parallel Programming (eBook)

Front Cover 1
Practical Parallel Programming 4
Copyright Page 5
Table of Contents 8
Dedication 6
Preface 14
Section 1: Background 18
Chapter 1. Parallel Paradigms 20
1.1 Nature of the Algorithm 20
1.2 Parallel Work Distribution 23
1.3 The Models 24
1.4 Application of the Models 27
1.5 Parallel Paradigms and Entire Programs 27
1.6 Alternative Paradigms 28
1.7 Silicon Graphics Implementations 29
1.8 Summary 30
Chapter 2. The Hardware and Operating System Environment 32
2.1 The Hardware Environment 32
2.2 Multiprocessing IRIX 35
2.3 Parallel Programming Support 40
2.4 Summary 44
Chapter 3. Pre-IRIS Power Language Parallel Programming 46
3.1 The Parallel Primitives 47
3.2 Summary 51
Chapter 4. Application of Parallel Primitives to Real Problems 52
4.1 p Approximation Using Homoparallelism 52
4.2 Dining Philosophers and Heteroparallelism 55
4.3 Summary 61
Section 2: IRIS Power-Language Prelude 62
Chapter 5. Data Dependence 64
5.1 Definitions 64
5.2 Dependence-Free Situations 66
5.3 Dependence Situations 67
5.4 Summary 71
Chapter 6. Parallelization of Loops 72
6.1 Homoparallelization 72
6.2 Scheduling 74
6.3 Managed Load Imbalance 76
6.4 Summary 79
Section 3: IRIS Power Languages 80
Chapter 7. Power Fortran Essentials 82
7.1 Compiling 82
7.2 Parallelization Strategy: Parallelized Do Loops 83
7.3 The c$doacross Directive 84
7.4 Auxiliary Parallel Directives 91
7.5 The Nature of Type Lastlocal 93
7.6 The Reduction Modifier 94
7.7 Conditional Parallel Execution of Loops 97
7.8 Parallel Execution Efficiency 98
7.9 Summary 105
Chapter 8. Power Fortran Toolbox 106
8.1 Power Fortran Parallelization Opportunities 106
8.2 Analysis of Individual Loops for Parallelization Opportunities 108
8.3 General Code Optimization 121
8.4 Summary 122
Chapter 9. Power Fortran Performance Issues 124
9.1 Factors Affecting Single-Program Performance 124
9.2 Features Affecting Multiple Program Performance 129
9.3 Resetting the Number of Active Threads 131
9.4 Summary 132
Chapter 10. Managing Fortran Data Dependence 134
10.1 Interthread Code Access Control: Locks 134
10.2 Flow-Dependence Control 137
10.3 Summary 140
Chapter 11. Advanced Features ofPower Fortran 142
11.1 Customization of the Parallel Environment 142
11.2 Sharing and Initializing Data Structuresin Parallel Regions 146
11.3 Power Fortran Implementation Details 154
11.4 Summary 156
Chapter 12. IRIS Power C Essentials 158
12.1 Compilation 158
12.2 IRIS Power C Parallel Fundamentals 159
12.3 Strategy of Parallelizing C Code: The Parallel Region 166
12.4 The for Loop 173
12.5 Independent Code Blocks 181
12.6 Summary 188
Chapter 13. Managing Data Dependence in IRIS Power C 190
13.1 Access Control to Code and Variables: #pragma critical 190
13.2 Temporary Serial Execution: pragma one processor 192
13.3 Tuning Access Control: #pragma lock 194
13.4 Flow or Execution Dependence 197
13.5 Summary 205
Chapter 14. IRIS Power C Toolbox 206
14.1 Identifying C Parallelization Opportunities 206
14.2 Individual Code Opportunities 208
14.3 Summary 242
Cahpter 15. IRIS Power C Performance Issues 244
15.1 if and ifinline Modifiers 244
15.2 Threads 246
15.3 Synchronization 246
15.4 Running in a Crowd: Executing a Parallel Program in a Multiprocess Environment 248
15.5 IRIS Power C Code Transformations 251
15.6 Summary 257
Section 4: Power Language Analyzers 258
Chapter 16. Power Fortran Accelerator Essentials 262
16.1 Power Fortran Accelerator Operations 262
16.2 PFA Advanced Code Handling Capabilities 269
16.3 PFA Directives and Assertions 274
16.4 Summary 282
Chapter 17. Power Fortran Accelerator Toolbox 284
17.1 Dependence Situations 284
17.2 Application of PFA Directives to Specific Code Situations 285
17.3 Summary 307
Chapter 18. PFA Advanced Features and Optimizations 308
18.1 Loop Scheduling 308
18.2 Controlling the PFA Analysis 309
18.3 Scalar Optimization: Loop Unrolling 311
18.4 Performance Management 311
18.5 Subprogram Inline Expansion 316
18.6 Parallel Portability 316
18.7 Summary 318
Chapter 19. Power C Analyzer Basics 320
19.1 Introduction 321
19.2 Limitations 321
19.3 Those Intermediate and Listing Files 322
19.4 Examples 322
19.5 Data Dependences 346
19.6 Pointer Notation 347
19.7 Controlling the PCA Analysis 350
19.8 Summary 356
Chapter 20. PCA Toolbox 358
20.1 Listing File Strategies 358
20.2 Compilation of Multi-Source File Programs 359
20.3 Individual Code Opportunities 360
20.4 Summary 379
Chapter 21. PCA Advanced Features and Optimizations 380
21.1 Controlling the Optimization Process 380
21.2 Analyzer Limits 382
21.3 Serial Optimizations 383
21.4 Controlling Parallelization 391
21.5 Optimization of Execution Performance 398
21.6 Compatibility with IRIS Power C 399
21.7 Problem Code Transformations 403
21.8 Inlining 405
21.9 Interprocedure Analysis 420
21.10 Summary 424
Section 5: Parallel Program Development Tools 426
Chapter 22. Parallel Program Development 428
22.1 Utility Programs 428
22.2 Profiling 428
22.3 Debugging 436
22.4 General Parallelization Strategies 437
22.5 Summary 440
Chapter 23. Problems in Practical Parallelization 442
23.1 C Language Example: The Knight's Tour 443
23.2 Fortran Example: Linpack 451
23.3 Summary 460
Bibliography 462
Index 464