High Performance Computing in Science and Engineering, Garching/Munich 2009 (eBook)

Preface 5
Contents 7
Part I Computer Science 13
Complexities of Performance Prediction for Bandwidth-Limited Loop Kernels on Multi-Core Architectures 14
Introduction 14
Experimental Test Bed 15
Bandwidth 16
Memory Bandwidth Balance Model 16
Limitations of the Memory Balance Model 16
Diagnostic Performance Model for Bandwidth-Limited Loop Kernels 18
Conclusion and Outlook 23
References 23
Performance Limitations for Sparse Matrix-Vector Multiplications on Current Multi-Core Environments 24
Introduction 24
Common Storage Schemes 26
Test Bed 27
Limitations of Serial Performance 28
Basic Sparse Vector Operations 28
Resulting Performance for SpMVM 32
Shared-Memory Parallel SpMVM 34
Intra-Socket Performance 34
Inter-Socket Performance 35
HLRB-II Scalability 36
Conclusion and Outlook 36
References 37
waLBerla: Optimization for Itanium-based Systems with Thousands of Processors 38
Introduction 38
The waLBerla Framework 39
Lattice Boltzmann Method 40
Single Phase Lattice Boltzmann Method 40
Free Surface Extension 41
Localized Bubble Merge Algorithm 42
Large-Scale Free Surface Applications 44
Itanium-Specific Optimizations 45
Conclusion 48
References 48
Fast 3D Block Parallelisation for the Matrix Multiplication Prefix Problem 50
Introduction: The Prefix Problem 50
Scope and Organisation of the Paper 51
Hardware and Software Setup 51
Parallelising the Prefix Problem 52
Coarse-Grain versus Fine-Grain Approach 52
3D Block-Oriented Parallel Matrix Multiplication 53
Numerical Results 55
Application: Optimal Quantum Control 56
Linear Algebra Tasks in the GRAPE Algorithm 58
Numerical Results 59
Conclusions 60
References 61
OMI4papps: Optimisation, Modelling and Implementation for Highly Parallel Applications 62
Performance Modelling Using the Apex-MAP Benchmark 62
The Apex-MAP Benchmark 63
Comparison of Apex-MAP with Real Application Performance 65
Modelling LRZ's Application Mix 66
Validation Using the EuroBen Mathematical Kernels 68
mod2am: Dense Matrix-Matrix Multiplication 69
mod2as: Sparse Matrix-Vector Multiplication 70
Validation of Apex-MAP 70
Conclusion and Outlook 71
References 73
Computational Steering of Complex Flow Simulations 74
Introduction 75
Computational Steering Environment 75
Hierarchical Approach 76
iFluids 76
Adaptive Load Balancing 77
Remote Visualisation and Steering Framework 78
Related Applications 80
Peano 81
Thermal Comfort Assessment 82
Summary and Outlook 84
References 85
Part II Computational Fluid Dynamics 86
Numerical Experiments for Quantification of Small-Scale Effects in Particle-Laden Turbulent Flow 87
Introduction 87
Numerical Simulation of the Carrier Flow 88
Discrete Particle Simulation 89
Validation of the Code 90
Effect of the SGS Turbulence on the Kinetic Energy Seen by the Particles 91
Effect of the SGS Turbulence on Preferential Concentration 94
Computational Requirements 95
Conclusions 96
References 97
On the Turbulence Structure in a Supersonic Diffuser with Circular Cross-Section 99
Introduction 99
Computational Details 100
Results 101
Conclusions 106
References 106
Numerical Simulation of Supersonic Jet Noise with Overset Grid Techniques for Highly Parallelized Computing 108
Introduction 108
Numerical Methods 110
Nozzle Geometry with Overset Grid Techniques 111
Results 114
Nozzle Geometry (Inlet 2) 114
Conclusion 116
References 116
Vorticity Statistics in Fully Developed Turbulence 118
Introduction 118
Vorticity Statistics in Homogeneous Isotropic Turbulence 119
Theoretical Framework 121
The Lundgren-Monin-Novikov Hierarchy 121
Statistical Symmetries 123
Numerical Methods and Computations 123
Algorithm 123
Typical Runs 124
Scaling and Performance 125
Merging Theory and Direct Numerical Simulations 126
Conclusions and Outlook 128
References 129
Assessment of CFD Predictive Capability for Aeronautical Applications 130
Project Description 130
Theory 131
The k- SST Turbulence Model 131
The SAS-SST Turbulence Model 131
Verification of URANS Predictions 132
Fan-in-Wing 133
Flow Phenomena Around a Helicopter Fuselage 137
Conclusions 140
References 141
Computational Aspects of Implicit LES of Complex Flows 142
Project Description 142
Numerical Method 144
Underlying Finite-Volume Discretization and ALDM 144
Body-Fitted Grid Simulation Code LESOCC2 145
Cartesian Grid Simulation Code INCA 145
Test Case Round Cylinder 148
Computational Setup 148
Simulation Results 149
Computational Aspects 151
Parallelization 151
Computational Effort 152
Computational Performance of CIIM 153
Conclusions 154
References 154
Numerical Investigation of the Micromechanical Behavior of DNA Immersed in a Hydrodynamic Flow 156
Description of the Research Project 156
Stretching of DNA Molecules in a Hydrodynamic Flow 156
Numerical Methods for DNA in a Hydrodynamic Flow 157
Model 157
Mesoscopic Modeling of the Solvent 158
Mechanical Modeling of the Polymer Chain 159
Object-Oriented Package sph2000 159
A Client for Parallel Particle Mesh (PPM) Library 160
Visualization 161
Summary of Results 162
Transport Properties of the Solvent 162
Static Properties of Polymer 163
Dynamic Properties of the Polymer 165
Confined Polymer and Polymer in Complex Flows 166
Conclusions 167
References 167
Comparing Frequency-Based Flow Solutions to Traditional Unsteady Fluid Dynamics Analysis in Turbomachinery 170
Introduction 170
Investigated Compressor Stage 172
Numerical Model 172
Full-Annulus and NLH Coarse Grid 173
NLH Fine Grid 173
Boundary Conditions and Numerical Settings 174
Comparison of Numerical Effort 174
Convergence Behaviour 175
Numerical Results 175
Compressor Maps 175
Blade-to-Blade-Flow 177
Comparison with Fourier Analysis 179
Conclusion 182
References 183
Capability of FDEM for Journal Bearings with Microstructured Surface 184
Introduction 184
Topic 186
Numerical Results 186
Results for Journal Bearings with Microstructured Surface 187
Scalability Tests 190
Concluding Remarks 191
References 192
Numerical Investigation of a Transonic Axial Compressor Stage with Inlet Distortions 193
Introduction 193
Description of the Test Case 194
Flow Solver Information 195
Type of Simulation 196
Results of Steady and Unsteady Simulations 197
Outlook 201
References 202
A Parallel CFD Solver Using the Discontinuous Galerkin Approach 204
Introduction 204
Discontinuous Galerkin Schemes 205
Basic Equations 205
Turbulence Modeling 205
Detached Eddy Simulation 206
Computational Aspects 206
Parallelisation and Object-Oriented Design 206
Speedup and Performance 207
Results 209
Flat Plate Flow 209
Detached Eddy Investigation of the Flow Past a Sphere 210
Conclusion 211
References 211
Characterization of the Aeroacoustic Properties of the SOFIA Cavity and its Passive Control 213
Introduction 214
Numerical Methods 215
Impact of Cavity Door Position on the Aeroacoustic Characteristics 216
Shear-Layer Control by Means of a Porous Fence 217
Impact of TA/AA Misalignment on Aeroacoustics 218
Parametric Optimization of the Aperture-Ramp Geometry 221
Conclusions 223
References 223
Towards the Numerical Simulation of a Scram Jet Intake at High Mach Number 225
Introduction 225
Goals of the Project 226
Numerical Code 226
Space-Time Expansion 228
Local Time Stepping 229
Scale-up Efficiency 229
Preparatory Calculations 230
3D Flow Around a Sphere 230
3D Freestream Injector 230
3D Turbulent Flow Around a Cylinder 232
Code Performance 233
Outlook 235
References 235
Direct Numerical Simulations of Turbulent Mixed Convection in Enclosures with Heated Obstacles 236
Introduction 237
Governing Equations and Computational Domain 237
Numerical Method and Mesh Resolution Requirements 239
Results 240
Details on Performance on the HLRB System 246
Conclusions 246
References 247
Determination of Acoustic Scattering Coefficients via Large Eddy Simulation and System Identification 248
Introduction 248
Background 249
Method: LES/SI 250
Results 252
Numerical Methods and High Performance Computing 255
Summary 257
References 258
Identification of Flame Transfer Functions Using LES of Turbulent Reacting Flows 260
Introduction 260
Background 261
Large Eddy Simulations 261
Combustion Model 262
The Flame Transfer Function 262
LES/SI 262
Numerical Set Up 263
Boundary Conditions 265
Results and discussion 266
Identification of Flame Transfer Function 268
Summary 269
References 270
Computational Modelling of the Respiratory System for Improvement of Mechanical Ventilation Strategies 272
Introduction 273
Research Software Platform BACI 273
Computational Model of the Central Airways 274
Computational Model of Lung Parenchyma 277
Summary and Outlook 280
References 281
Part III Geo Sciences 283
SeisSol -- A Software for Seismic Wave Propagation Simulations 284
Introduction 285
The Numerical Approximation 285
Non-conforming Hybrid Meshes 286
Highly Heterogeneous Material 288
Large-Scale Basin Application 289
Scalability 292
Concluding Remarks 293
References 294
Advances in Modelling and Inversion of Seismic Wave Propagation 296
Introduction 296
Topographic Effects on Seismic Waves 298
Seismic Tomography Using Waveforms 300
Time Reversal of Seismic Waves 302
Scattering in the Earth's Crust 304
Conclusions 307
References 308
Part IV Astrophysics 310
Constrained Local UniversE Simulations (CLUES) 311
Introduction 311
Constrained Simulations 313
Observational Data 313
Constrained Initial Conditions 313
Description of Simulations 314
An Ensemble of Constrained Simulations 317
Latest Results from CLUES Simulations 319
Warm Dark Matter in the Local Universe 319
Satellites in the Local Group 322
Summary 323
References 324
The Core Helium Flash: 3D Hydrodynamic Models 325
Introduction 325
Initial Model 326
Code 327
Results 328
Conclusions 333
References 335
3D Simulations of Large-Scale Mixing in Core Collapse Supernova Explosions 337
Introduction 338
Simulation Setup 339
Code Performance 340
Results 340
Dynamic Evolution 340
Radial Element Mixing 343
Conclusions 345
References 347
Relativistic Simulations of Neutron Star and Strange Star Mergers 349
Introduction 349
Mathematical Model and Numerical Implementation 350
Scalability of the Code 352
Simulations and Results 353
Summary, Conclusions and Outlook 359
References 359
The Physics of Galactic Nuclei 361
Introduction 361
Nuclear Disc Formation in Galactic Nuclei 362
Numerical Method and Model Setup 362
Density Evolution 364
Accretion 364
Disc Properties 365
Radiation Pressure Driven Dust Cloud Interactions 366
Multi-Phase Turbulence in the Tori of Active Galactic Nuclei 367
Feeding Supermassive Black Holes with Nuclear Star Clusters 369
Numerics and Code Performance 370
References 371
Numerical Models of Turbulence in Isothermal and Thermally Bistable Interstellar Gas 373
Introduction 373
Large Eddy Simulation 374
Supersonic Isothermal Turbulence 375
Turbulence Energy 376
Turbulent Viscosity 377
Turbulent Dissipation 379
Thermally Bistable Turbulence 380
Conclusion 382
References 383
Turbulence Modeling and the Physics of the Intra-Cluster Medium 385
Introduction 386
Numerical Tools for the Modeling of Turbulent Flows 387
Resolving the Turbulent Flow with AMR 388
FEARLESS Simulation of a Galaxy Cluster 391
Conclusions and Outlook 394
References 395
Project h1021: Dynamics of Binary Black Hole Systems 397
Goals and Motivation 397
Model and Methodology 398
Computational Infrastructure 399
Performance and Scaling 400
Resources Required for Typical Simulations 404
Status Report for h1021 405
Conclusion 408
References 408
Sheared Magnetic Field and Kelvin-Helmholtz Instability 410
Introduction 410
Model and Numerical Methods 411
The Model 411
Numerical Methods 411
Results 412
Single Layer Simulations 412
Slab-Jet Simulation 413
References 415
Solar Surface Flow Simulations at Ultra-High Resolution 416
Introduction 417
Current Status of the Simulations 418
Results 418
Applications and Interpretation 420
Visualization 423
Next Project Stages and Conclusions 424
References 425
Part V High-Energy Physics 427
Lattice Investigation of Nucleon Structure: Towards the Physical Point 428
Introduction 428
The Simulation 429
Nucleon Structure at Light Quark Masses 430
Electromagnetic Form Factors 431
Moments of Structure Functions 433
Conclusion and Outlook 435
References 436
Dynamical Lattice QCD with Ginsparg-Wilson-Type Fermions 438
The Research Field and Our Research Strategy 439
Investigations with Dynamical Chirally Improved Quarks 440
Generation of Ensembles 440
Ground State and Excited Hadron Masses 441
Filtering and Topological Charge Densities 442
2+1 Flavour QCD Results Obtained with the Fixed-Point Action 444
The -regime for Different Topological Sectors 444
References 447
Continuum-Limit Scaling of Chirally Symmetric Fermions as Valence Quarks 450
Introduction 450
A Brief Review of Overlap Fermions 451
The Need for Overlap Fermions 451
Techniques to Effectively Deal with Overlap Fermions 453
Simulation Setup 456
Computational Details 456
Parallelisation 456
Timings 457
Results 457
Tree-Level Test 457
The Interacting Case -- Matching the Pion Mass 457
Continuum-Limit Scaling of the Pion Decay Constant 459
Conclusion and Prospects 460
References 460
Quantum Boltzmann Equations in the Early Universe 462
Introduction 462
Kadanoff-Baym Equations 463
Numerical Results 465
Numerical Methods 467
Single-Host Algorithm 468
Parallel Distributed-Memory Algorithm 470
Memory Requirements 472
Conclusions and Outlook 472
References 473
Topological Structure of the QCD Vacuum Revealed by Overlap Fermions 474
Introduction: Overlap Fermions and Topological Charge 475
Topological Density with Different Resolution 476
Cluster Analysis 477
Fractal Dimensions 479
Smearing vs. Filtering 480
Selfduality 481
Localization and Local Chirality of Overlap Eigenmodes 483
Technical Details 483
Conclusions 484
References 485
Part VI Condensed Matter Physics 487
Gyrokinetic Turbulence Investigations Involving Ion and Electron Scales 488
Introduction 488
Magnetic Confinement Fusion and Plasma Turbulence 488
Plasma Turbulence Investigations Using Gyrokinetic Theory 489
The Plasma Turbulence Code Gene 490
Nonlinear Gyrokinetic Simulations Covering Multiple Spatio-Temporal Scales 491
Introduction and Context 491
Simulation Details 492
Simulation Results 493
ETG Turbulence in Edge Transport Barriers 495
High- Simulations and Microturbulence in Astrophysics 496
Conclusions 497
References 497
Quantum Monte Carlo Studies of Strongly Correlated Electron Systems 499
Magnetic Field Induced Semimetal-to-Canted-Antiferromagnet Transition on the Honeycomb Lattice 500
CTQMC Study of the Single Impurity and Periodic Anderson Models with s-Wave Superconducting Baths 504
Quantum Dot with Two Superconducting Baths 504
Periodic Anderson Model with Superconducting Conduction Band 507
Accessing the Thermodynamic Properties in the Hubbard Model 509
Method and Numerical Aspects 509
References 511
Deacon Process over RuO2 and TiO2-Supported RuO2 513
Introduction 514
Calculational Details 514
Reaction Mechanism for the Chlorination of RuO2(110) 516
Reaction Mechanism of the HCl Oxidation over RuO2(110) 518
Reaction Mechanism of the HCl Oxidation over RuO2(110) Supported on TiO2(110): DFT Predictions 520
Concluding Remarks 522
References 523
Charge-Carrier Transport Through Guanine Crystals and Stacks 525
Introduction 525
Theory, Computational Method, and Performance 528
Results and Discussion 530
Conclusions 534
References 535
Nanomagnetism in Transition Metal Doped Si Nanocrystals 537
Introduction 537
Computational Methods 538
DFT Framework 538
Numerical Simulations and Performance 539
Modeling and Methods 540
Nanocrystal Construction 540
Approaches Beyond GGA 542
Results and Discussion 542
Stability and Geometry 542
Self-purification Effect 545
Electronic and Magnetic Properties of Subsurface Doping Sites 546
Summary 547
References 547
High Performance Computing for the Simulation of Thin-Film Solar Cells 549
Introduction 549
Simulation of Thin-Film Solar Cells 550
Modeling Maxwell's Equations 551
Solar Cell Model 554
Integration of AFM Scans 555
Parallelization 556
Simulation Results 558
Conclusion 559
References 560
Origin of Interface Magnetism in Fe2O3/FeTiO3 Heterostructures 561
Introduction 561
Method and Details of the Calculations 563
Results and Discussion 563
Structural Relaxation 564
Energetic Stability 565
Electronic Properties 566
Influence of Epitaxial Strain 566
Summary 569
References 569
Evaluation of Magnetic Spectra Using the Irreducible Tensor Operator Approach 571
Introduction 571
Goals of the Project 572
Scientific Results from the HLRB Computations 573
Basic Background 573
General Point-Group Symmetries 573
Result I: Complete Energy Spectra of Frustrated Magnetic Molecules 575
Result II: Approximate Energy Spectra of Magnetic Molecules 577
Computations Run on the HLRB II 579
Technical and Algorithmic Methods 579
Programming Techniques 580
Technical Results from the HLRB Computations 580
Scaling of Calculations Using the Irreducible Tensor Operator Technique 580
Scaling of our Lanczos Diagonalization Routine 581
Technical and Numerical Issues for the Future 582
References 582
Simulating Strongly Coupled Plasmas on High-Performance Computers 585
Introduction to the Physics of Strongly Coupled Plasmas 585
Resolving Different Length Scales in the Simulation of Strongly Coupled OCPs 587
Resolving Different Time Scales in the Simulation of Strongly Coupled OCPs 587
Numerical Techniques used in Integrating the Equation of Motion 588
Recent Results on Stopping Highly Charged Ions in a Strongly Coupled Ion Plasma 590
Conclusion and Outlook 593
References 593
Material-Specific Investigations of Correlated Electron Systems 595
Introduction 595
Computational Method 596
The LDA+DMFT Approach 596
QMC Method 597
Results and Discussion 598
Pressure-Driven Metal-Insulator Transition in Hematite 598
Fluctuating Valence and Valence Transition of Yb Under Pressure 601
Metal-Insulator Transition in NiS2-xSex 602
Interaction Driven Insulator-to-Insulator Transition 604
Conclusions 605
References 606
Theoretical Study of Electron Transfer and Electron Transport Processes in Molecular Systems at Metal Substrates 609
Introduction 609
Photoinduced Electron Transfer of Molecules at Surfaces 610
Electron-Transfer Hamiltonian 610
Determination of Model Parameters 611
Electronic Structure Calculations 612
Electron Dynamics in Benzonitrilethiolate at Au(111) 613
Computational Details 613
Electron Transport in Single-Molecule Junctions 614
Electron Transport Theory 614
Determination of Model Parameters 616
Vibrational Nonequilibrium Effects in Electron Transport Through Benzenedibutanethiolate 616
Photoinduced Switching of a Molecular Junction via Hydrogen Transfer 618
Computational Details 619
Concluding Remarks 620
References 621
Fluctuations in the Photoionization Cross Sections of Highly Doubly Excited Two-Electron Atoms 623
Introduction 623
Theory and Numerical Implementation 625
Results 627
Summary and Outlook 631
References 632
Part VII Chemistry 634
Photophysics of the Trp-Gly Dipeptide: Role of Electron and Proton Transfer Processes for Efficient Excited-State Deactivation 635
Introduction 635
Computational Methods 636
Scientific Results 637
Ground-State Equilibrium Structure 637
Excitation Energies and Molecular Orbitals 638
Proton-Transfer Reaction Path 638
Discussion of Reaction Mechanisms 640
Conclusions 640
Technical Results 641
References 642
Grid Workflows for Molecular Simulations in Chemical Industry 644
Introduction 644
Workflows for VLE Simulations 646
The GridSFEA Framework 647
Integration of GridSFEA with Workflow Management System 649
Workflow Management Systems Review 649
WS-VLAM Organisation and Properties 650
Integration of GridSFEA with WS-VLAM 651
Results 653
Concluding Remarks 654
References 655
Global Chemistry-Climate Modelling with EMAC 656
Introduction 656
Technical Information 658
Model Development 659
Upper-Boundary Parameterisation 659
Decoupling of Dynamics and Chemistry 660
Atmosphere-Ocean Feedbacks 661
Lagrangian Modelling 662
Production-Oriented Simulations 663
Atmospheric Ice Nuclei 663
Impact of Ship Emissions on Atmospheric Composition and Climate 665
Final Remarks 666
References 667
Ab Initio Path Integral Simulations of Floppy Molecular Systems 668
Introduction 668
Simulation Techniques and Technical Details 670
Results 671
Static Properties 671
Radii of Gyration 673
Analysis of CH5+ 676
Performance and Scaling 676
Outlook and Planned Work 677
References 678
Statistically Converged Properties of Water from Ab Initio Molecular Dynamics Simulations 680
Introduction 680
Methods 684
Results 686
Discussion & Outlook
References 690
Ab Initio Molecular Dynamics Simulations of Aqueous Glycine Solutions: Solvation Structure and Vibrational Spectra 692
Introduction 692
Methods 693
Results and Discussion 695
Conclusions and Outlook 699
References 699
Cyclodimerization of DNA and RNA Bases: Ab Initio Study of the Cyclodimerization of the Uracil Dimer Through a Butane-Like Conical Intersection 702
Introduction 702
Computational Methods 704
Results and Discussion 705
Geometry Optimization 705
Vertical Excitation Energies 706
Reaction-Path Energy Profiles 707
Conclusions 709
Technical Support 709
References 710
Numerical Simulation of Electric Field Gradient Focusing and Separation of Analytes in Microchannels with Embedded Bipolar Electrode 712
Introduction 712
Experimental Section 713
Theoretical Background 715
Numerical Methods 718
Results and Discussion 719
Conclusion 722
References 723
Part VIII Bio Sciences 724
Annotation of Entirely Sequenced Genomes 725
Methods 726
Iterated Profile-Based Search (PSI-BLAST) 726
Functional Sequence Motifs (ProSite) 726
Low-Complexity Regions (SEG) 726
Domain Assignment (ProDom) 727
Secondary Structure (PHDsec) 727
Solvent Accessibility (PHDacc) 727
Globularity of Proteins (GLOBE) 728
Transmembrane Helices (PHDhtm) 728
Secondary Structure (PROFsec) and Solvent Accessibility (PROFacc) 728
Coiled-Coil Regions (COILS) 729
Disulphide Bridges (DISULFIND) 729
Structural Switches (ASP) 729
Localisation Classification (LOCtree) 729
Prediction of Nuclear Localisation Signal (PredictNLS) 730
Keyword Based Prediction of Cellular Localization (LOCkey) 730
Prediction of Unstructured Loops (NORSnet) 730
Prediction of Natively Unstructured Regions Through Contacts (Ucon) 731
Meta-Disorder Predictor (MD) 731
Prediction of Flexibility (PROFbval) 731
Reflect 731
Results and Discussion 733
Parallelization Techniques 733
Number of Processors Used/Degree of Parallelization 734
Job Run Times 735
Conclusion 735
References 736
Molecular Dynamics Simulation of the Nascent Peptide Chain in the Ribosomal Exit Tunnel 738
Introduction 738
Methods 740
Principles of Molecular Dynamics 740
Periodic Boundary Condition 742
Temperature and Pressure Coupling 742
Parallelisation Techniques 743
Preparation of Simulation System 743
Results 744
Achieved Performance on the HLRB system 744
Conformations of Polypeptides Inside the Tunnel 744
References 746
Preparing RAxML for the SPEC MPI Benchmark Suite 747
Introduction 747
The Phylogenetic Likelihood Kernel 749
Sources of Parallelism in Phylogenetic Analyses and Related Work 750
Efficient Parallelization of the Phylogenetic Likelihood Kernel 751
Master/Worker Scheme 752
Experimental Setup and Results 754
Conclusion 756
References 757
Parallel Computing with the R Language in a Supercomputing Environment 759
Introduction 759
R and Parallel Computing 760
R at the HLRB2 761
Benchmark 763
Applications 764
Indirect Comparison of Interaction Graphs 764
Parallel Computing in Microarray Data 766
Conclusion 768
References 769