Stochastic Optimization Methods (eBook)

Dr. Kurt Marti is a full Professor of Engineering Mathematics at the „Federal Armed Forces University of Munich“. He is Chairman of the IFIP-Working Group 7.7 on “Stochastic Optimization” and has been Chairman of the GAMM-Special Interest Group “Applied Stochastics and Optimization”. Professor Marti has published several books, both in German and in English, and he is author of more than 160 papers in refereed journals.

Preface 5
Contents 9
Part I Basic Stochastic Optimization Methods 15
1 Decision/Control Under Stochastic Uncertainty 16
1.1 Introduction 16
1.2 Deterministic Substitute Problems: Basic Formulation 18
2 Deterministic Substitute Problems in Optimal Decision Under Stochastic Uncertainty 22
2.1 Optimum Design Problems with Random Parameters 22
2.2 Basic Properties of Substitute Problems 31
2.3 Approximations of Deterministic Substitute Problems in Optimal Design 32
2.4 Applications to Problems in Quality Engineering 42
2.5 Approximation of Probabilities: Probability Inequalities 43
Part II Differentiation Methods 54
3 Differentiation Methods for Probability and Risk Functions 56
3.1 Introduction 56
3.2 Transformation Method: Differentiation by Using an Integral Transformation 59
3.3 The Differentiation of Structural Reliabilities 67
3.4 Extensions 70
3.5 Computation of Probabilities and its Derivatives by Asymptotic Expansions of Integral of Laplace Type 75
3.6 Integral Representations of the Probability Function P(x) and its Derivatives 85
3.7 Orthogonal Function Series Expansions I: Expansions in Hermite Functions, Case m = 1 88
3.8 Orthogonal Function Series Expansions II: Expansions in Hermite Functions, Case m> 1
3.9 Orthogonal Function Series Expansions III: Expansions in Trigonometric, Legendre and Laguerre Series 104
Part III Deterministic Descent Directions 107
4 Deterministic Descent Directions and E.cient Points 108
4.1 Convex Approximation 108
4.2 Computation of Descent Directions in Case of Normal Distributions 114
4.3 Efficient Solutions (Points) 126
4.4 Descent Directions in Case of Elliptically Contoured Distributions 131
4.5 Construction of Descent Directions by Using Quadratic Approximations of the Loss Function 134
Part IV Semi-Stochastic Approximation Methods 140
5 RSM-Based Stochastic Gradient Procedures 142
5.1 Introduction 142
5.2 Gradient Estimation Using the Response Surface Methodology ( RSM) 144
5.3 Estimation of the Mean Square (Mean Functional) Error 155
5.4 Convergence Behavior of Hybrid Stochastic Approximation Methods 160
5.5 Convergence Rates of Hybrid Stochastic Approximation Procedures 166
6 Stochastic Approximation Methods with Changing Error Variances 190
6.1 Introduction 190
6.2 Solution of Optimality Conditions 191
6.3 General Assumptions and Notations 192
6.4 Preliminary Results 196
6.5 General Convergence Results 203
6.6 Realization of Search Directions 217
6.7 Realization of Adaptive Step Sizes 233
6.8 A Special Class of Adaptive Scalar Step Sizes 249
Part V Reliability Analysis of Structures/Systems 264
7 Computation of Probabilities of Survival/ Failure by Means of Piecewise Linearization of the State Function 266
7.1 Introduction 266
7.2 The State Function 269
7.3 Probability of Safety/Survival 272
7.4 Approximation I of ps, pf: FORM 275
7.5 Approximation II of ps, pf: Polyhedral Approximation of the Safe/Unsafe Domain 283
7.6 Computation of the Boundary Points 292
7.7 Computation of the Approximate Probability Functions 295
Part VI Appendix 312
A Sequences, Series and Products 314
A.1 Mean Value Theorems for Deterministic Sequences 314
A.2 Iterative Solution of a Lyapunov Matrix Equation 322
B Convergence Theorems for Stochastic Sequences 326
B.1 A Convergence Result of Robbins–Siegmund 326
B.2 Convergence in the Mean 329
B.3 The Strong Law of Large Numbers for Dependent Matrix Sequences 331
B.4 A Central Limit Theorem for Dependent Vector Sequences 332
C Tools from Matrix Calculus 334
C.1 Miscellaneous 334
C.2 The v. Mises-Procedure in Case of Errors 335
References 340
Index 348