Stochastic Global Optimization and Its Applications with Fuzzy Adaptive Simulated Annealing (eBook)

Title Page 
1 
Preface 5
Contents 8
Part I: Fundamentals 12
Introduction 13
Why to Optimize? 13
Kinds of Optimization Problems 15
How to Optimize? 16
References 20
Global Optimization and Its Applications 21
Introduction 21
Stochastic or Deterministic ? 22
Considerations about General Global Optimization Tasks 23
Some Popular Approaches and Final Comments 28
References 30
Metaheuristic Methods 31
Introduction 31
Genetic Algorithms 33
Particle Swarm Optimization 34
Differential Evolution 35
Cross-Entropy Method 36
Simulated Annealing 37
References 40
Part II: ASA, Fuzzy ASA and Their Characteristics 41
Adaptive Simulated Annealing 42
Introduction 42
LICENSE and Contributions 43
Organization of Chapter 43
Theoretical Foundations of Adaptive Simulated Annealing (ASA) 44
Shades of Simulated Annealing 44
Critics of SA 45
``Standard'' Simulated Annealing (SA) 45
Boltzmann Annealing (BA) 45
Simulated Quenching (SQ) 48
Fast Annealing (FA) 49
Adaptive Simulated Annealing (ASA) 49
VFSR and ASA 53
Practical Implementation of ASA 53
Generating Probability Density Function 53
Acceptance Probability Density Function 54
Reannealing Temperature Schedule 54
QUENCH_PARAMETERS=FALSE 55
QUENCH_COST=FALSE 56
QUENCH_COST_SCALE=TRUE 56
Tuning Guidelines 56
The Necessity for Tuning 56
Construction of the Code 57
Motivations for Tuning Methodology 59
Some Rough But Useful Guidelines 59
Quenching 61
Options for Large Spaces 62
Shunting to Local Codes 63
Judging Importance-Sampling 64
User References 64
Adaptive OPTIONS 65
VFSR 65
ASA_FUZZY 65
Multiple Systems 65
SELF_OPTIMIZE 65
ASA_PARALLEL 66
TRD Example of Multiple Systems 66
Conclusion 67
References 68
Unconstrained Optimization 72
Fuzzy ASA 72
Unconstrained (or Rectangular Constrained) Optimization Examples 76
Rastrigin Function 79
Schwefel Function 82
Ackley Function 85
Krishnakumar Function 87
Rosenbrock Function 89
Griewangk Function 92
Special Function 1 94
Special Function 2 97
Conclusion 101
References 102
Constrained Optimization 103
Introduction 103
Constrained Global Optimization Using ASA and Fuzzy ASA 105
Function G01 106
Function G02 110
Function G03 113
Function G04 114
Function G05 115
Function G06 115
Function G07 116
Function G08 117
Function G09 118
Function G10 119
Function G11 120
Function G12 121
Function G13 121
Conclusion 122
References 123
Part III Applications 124
Applications to Signal Processing - Blind Source Separation 125
Introduction 125
Implementation 130
Results 130
Example 1 - Separation by TSK MIMO System 130
Example 2 - Separation by TSK MIMO System 133
Example 3 - Separation by TSK MIMO System 134
Example 4 - Separation by TSK MIMO System 135
Example 5 - Mixture by PNL Model 138
Conclusion 143
References 144
Fuzzy Modeling with Fuzzy Adaptive Simulated Annealing 145
Introduction 145
Affine Takagi-Sugeno Fuzzy Systems 146
The Fuzzy Modeling Problem 147
Approximation in Lower Dimensions 147
Approximation in Higher Dimensions 150
Ideas for Fuzzy Clustering Using ASA 151
Conclusions about the Presented Methods 153
References 154
Statistical Estimation and Global Optimization 155
Introduction 155
Maximum Likelihood Estimation with ASA 156
Implementation and Experiments 157
Exponential Distribution 158
Normal Distribution 162
Lognormal Distribution 163
Cauchy Distribution 164
Triangular Distribution 166
Mixture (Laplace and Uniform) Distribution 170
Gamma Distribution 170
Conclusions 172
References 173
Nonlinear Equation Solving 174
Introduction 174
Statement of the Problem 175
The Algorithm 176
Examples 177
Example 1 177
Example 2 181
Example 3 183
Example 4 186
Example 5 186
Example 6 187
Example 7 189
Conclusions 189
References 192
Space-Filling Curves and Fuzzy ASA 193
Introduction 193
Key Results from General Topology, Ergodic and Measure Theories 194
Composing Space-Filling Curves and ASA 200
Algorithm Description 200
Experiments 201
Conclusions 203
References 205
Epilogue 206
Final Thoughts 206
Index 208