Tree-Structure based Hybrid Computational Intelligence (eBook)

Preface 7
Contents 10
Part I Foundations of Computational Intelligence 14
1 Foundations of Computational Intelligence 15
1.1 Introduction 15
1.2 Evolutionary Algorithms 15
1.2.1 Genetic Programming 19
1.2.2 Estimation of Distribution Algorithm 23
1.2.3 Population-Based Incremental Learning 25
1.2.4 Probabilistic Incremental Program Evolution 26
1.3 Swarm Intelligence 30
1.3.1 Particle Swarm Optimization 31
1.3.2 Ant Colony Optimization 33
1.4 Artificial Neural Networks 35
1.4.1 Architecture and Learning Algorithm 36
1.4.2 Multilayer Perceptron 38
1.4.3 Back-Propagation Algorithm 38
1.4.4 Evolutionary Algorithm Based Training 40
1.4.5 Self Organizing Feature Maps 41
1.4.6 Radial Basis Function 42
1.4.7 Recurrent Neural Networks 42
1.4.8 Adaptive Resonance Theory 43
1.5 Fuzzy Systems 43
1.5.1 The Definition of Fuzzy Sets 43
1.6 Takagi-Sugeno Fuzzy Model 44
1.6.1 Universal Approximation Property 45
1.6.2 Fuzzy Expert Systems - Design Challenges 45
1.7 Probabilistic Computing 46
1.8 Hybrid Intelligent Systems 47
1.9 Models of Hybrid Intelligent Systems 48
Part II Flexible Neural Trees 49
2 Flexible Neural Tree: Foundations and Applications 50
2.1 Introduction to Flexible Neural Tree 50
2.2 Flexible Neural Tree Algorithms 51
2.2.1 Encoding and Evaluation 51
2.2.2 Flexible Neuron Instructor 51
2.2.3 Fitness Function 53
2.2.4 Structure and Parameter Learning 53
2.2.5 Flexible Neural Tree Applications 55
2.2.6 Exchange Rate Forecasting 75
2.2.7 Face Recognition 80
2.2.8 Microarray-Based Cancer Classification 84
2.2.9 Protein Fold Recognition 87
2.3 Multi Input Multi Output Flexible Neural Tree 90
2.4 Representation and Calculation of the MIMO FNT 91
2.4.1 Hybrid Algorithm for Structure and Parameter Learning 93
2.4.2 Hybrid Algorithm for Flexible Neural Tree Model 95
2.4.3 Illustrative Examples 95
2.5 Ensemble of Flexible Neural Tree 100
2.5.1 The Basic Ensemble Method 101
2.5.2 The Generalized Ensemble Method 101
2.5.3 The LWPR Method 101
2.5.4 Stock Index Forecasting Problem 102
2.6 Stock Index Forecasting Experimental Illustrations 104
Part III Hierarchical Neural Networks 108
3 Hierarchical Neural Networks 109
3.1 Hierarchical Radial Basis Function Neural Networks 109
3.1.1 The Radial Basis Function Network 110
3.1.2 Automatic Design of Hierarchical Radial Basis Function 111
3.1.3 Tree Structure Optimization by Extended Compact 112
3.1.4 Parameter Optimization Using Differential Evolution 112
3.1.5 Procedure of The General Learning Algorithm 113
3.1.6 Variable Selection in the HRBF Network Paradigms 113
3.1.7 Experimental Illustrations 114
3.1.8 Face Recognition 115
3.2 Hierarchical B-Spline Neural Networks 118
3.2.1 The B-Spline Network 118
3.3 Automatic Design of HB-Spline Network 119
3.3.1 Encode and Calculation for HB-Spline 119
3.3.2 Tree Structure and Parameter Optimization 120
3.3.3 Procedure of the General Learning Algorithm 121
3.3.4 Variable Selection in the Hierarchical B-Spline Network 121
3.3.5 Experimental Illustrations 121
3.3.6 Wisconsin Breast Cancer Detection 121
3.3.7 Time-Series Forecasting 123
3.4 Hierarchical Wavelet Neural Networks 128
3.4.1 Wavelet Neural Network 128
3.5 Automatic Design of Hierarchical Wavelet Neural Network 129
3.5.1 Ant Programming for Evolving the Architecture of 129
3.5.2 Parameter Optimization Using Differential Evolution 131
3.5.3 Procedure of the General Learning Algorithm for HWNN 131
3.5.4 Variable Selection Using HWNN Paradigms 132
3.5.5 Experimental Illustrations 132
3.5.6 Application to Jenkins-Box Time-Series 134
Part IV Hierarchical Fuzzy Systems 136
4 Hierarchical Fuzzy Systems 137
4.1 Introduction 137
4.2 Takagi-Sugeno Fuzzy Inference System (TS-FS) 139
4.3 Hierarchical TS-FS: Encoding and Evaluation 139
4.3.1 Encoding 140
4.3.2 Evaluation 141
4.3.3 Objective Function 142
4.4 Evolutionary Design of Hierarchical TS-FS 143
4.4.1 Algorithm for Designing Hierarchical TS-FS Model 143
4.4.2 Feature/Input Selection with Hierarchical TS-FS 144
4.5 Experimental Illustrations 145
4.5.1 Systems Identification 146
4.5.2 Chaotic Time-Series of Mackey-Glass 147
4.5.3 Iris Data Classification 150
4.5.4 Wine Data Classification 152
Part V Reverse Engineering of Dynamical Systems 156
5 Reverse Engineering of Dynamic Systems 157
5.1 Introduction 157
5.2 Calculation and Representation of Additive Models 158
5.3 Hybrid Algorithm 159
5.3.1 Tree-Structure Based Evolutionary Algorithm 159
5.3.2 Evolving an Optimal or Near-Optimal Structure of 160
5.3.3 Parameter Optimization 162
5.3.4 Summary of General Learning Algorithm 164
5.3.5 Experimental Illustrations 165
5.3.6 Discussions 171
5.4 Inferring a System of Differential Equations 174
5.5 Inference of Differential Equation Models by Multi Expression Programming 175
5.5.1 Structure Optimization by the MEP 175
5.5.2 Parameter Optimization by Particle Swarm Optimization 176
5.5.3 Fitness Definition 177
5.5.4 Summary of Algorithm 178
5.6 Modeling Chemical Reactions 178
5.6.1 Simple Chemical Reaction Model 179
5.6.2 Two-Species Lotka-Volterra Model 180
5.6.3 Bimolecular Reaction 181
5.7 Inferring Gene Regulatory Networks 182
5.7.1 The Small Artificial Gene Regulatory Network 184
5.7.2 The Large-Scale Artificial Gene Regulatory Network with 187
Part VI Conclusions and Future Research 189
6 Concluding Remarks and Further Research 190
6.1 Limitations of Conventional Computational Intelligence 190
6.2 Towards Tree-Structure Based Hierarchical Hybrid Computational Intelligence 191
6.2.1 Tree Structure Based Evolutionary Computation Models 191
6.2.2 Hierarchical Hybrid Computational Intelligence 191
6.3 Static and Dynamical Models 195
References 196