Neural Networks (eBook)

Scientists

Preface 5
Reading Guide 6
Contents 8
List of Contributors 16
1 Neural Networks: An Overview 18
1.1 Neural Networks: Definitions and Properties 19
1.2 When and How to Use Neural Networks with Supervised Training 41
1.3 Feedforward Neural Networks and Discrimination ( Classification) 49
1.4 Some Applications of Neural Networks to Various Areas of Engineering 67
1.5 Conclusion 93
1.6 Additional Material 94
References 97
2 Modeling with Neural Networks: Principles and Model Design Methodology 101
2.1 What Is a Model? 101
2.2 Elementary Concepts and Vocabulary of Statistics 103
2.3 Static Black-Box Modeling 108
2.4 Input Selection for a Static Black-Box Model 111
2.5 Estimation of the Parameters (Training) of a Static Model 119
2.6 Model Selection 147
2.7 Dynamic Black-Box Modeling 165
2.8 Dynamic Semiphysical (Gray Box) Modeling 191
2.9 Conclusion: What Tools? 202
2.10 Additional Material 203
References 215
3 Modeling Methodology: Dimension Reduction and Resampling Methods 218
3.1 Introduction 218
3.2 Preprocessing 219
3.3 Input Dimension Reduction 222
3.4 Principal Component Analysis 222
3.5 Curvilinear Component Analysis 226
3.6 The Bootstrap and Neural Networks 235
References 245
4 Neural Identification of Controlled Dynamical Systems and Recurrent Networks 246
4.1 Formal Definition and Examples of Discrete-Time Controlled Dynamical Systems 247
4.2 Regression Modeling of Controlled Dynamical Systems 257
4.3 On-Line Adaptive Identification and Recursive Prediction Error Method 265
4.4 Innovation Filtering in a State Model 273
4.5 Recurrent Neural Networks 285
4.6 Learning for Recurrent Networks 291
4.7 Appendix (Algorithms and Theoretical Developments) 298
References 302
5 Closed-Loop Control Learning 303
5.1 Generic Issues in Closed-Loop Control of Nonlinear Systems 304
5.2 Design of a Neural Control with an Inverse Model 308
5.3 Dynamic Programming and Optimal Control 317
5.4 Reinforcement Learning and Neuro-Dynamic Programming 328
References 339
6 Discrimination 342
6.1 Training for Pattern Discrimination 343
6.2 Linear Separation: The Perceptron 347
6.3 The Geometry of Classification 349
6.4 Training Algorithms for the Perceptron 352
6.5 Beyond Linear Separation 368
6.6 Problems with More than two Classes 375
6.7 Theoretical Questions 377
6.8 Additional Theoretical Material 387
References 389
7 Self-Organizing Maps and Unsupervised Classification 391
7.1 Notations and Definitions 393
7.2 The k-Means Algorithm 395
7.3 Self-Organizing Topological Maps 404
7.4 Classification and Topological Maps 427
7.5 Applications 433
References 453
8 Neural Networks without Training for Optimization 455
8.1 Modelling an Optimisation Problem 455
8.2 Complexity of an Optimization Problem 458
8.3 Classical Approaches to Combinatorial Problems 459
8.4 Introduction to Metaheuristics 460
8.5 Techniques Derived from Statistical Physics 461
8.6 Neural Approaches 475
8.7 Tabu Search 496
8.8 Genetic Algorithms 496
8.9 Towards Hybrid Approaches 497
8.10 Conclusion 497
References 498
About the Authors 503
Index 505