Computational Intelligence – Synergies of Fuzzy Logic, Neural Networks and Evolutionary Computing

Nazmul Siddique is a lecturer in the School of Computing and Intelligent Systems at the University of Ulster. He has published over 120 scientific research papers in journals and conferences including seven book chapters and two books. He is a senior member of the IEEE and has been involved in organising many international conferences. He is on the editorial board of the International Journal of Neural Systems , International Journal of Automation and Control Engineering , Journal of Behavioural Robotics , and Engineering Letters . Hojjat Adeli is the holder of Abba G. Lichtenstein Professorship at The Ohio State University (OSU). He is the Editor-in-Chief of three journals: Computer-Aided Civil and Infrastructure Engineering , Integrated Computer-Aided Engineering , and International Journal of Neural Systems . He has authored over 500 publications including 14 books and has won numerous awards. He is a Distinguished Member of ASCE, a Fellow of AAAS and IEEE. In April 2010 he was profiled as an engineering legend in the journal Leadership and Management in Engineering .

Foreword vii Preface ix Acknowledgement xi Chapter 1: Introduction 1-20 1.1 Computational Intelligence 1 1.2 Paradigms of Computational Intelligence 2 1.3 Synergies of Computational Intelligence Techniques 11 1.4 Applications of Computational Intelligence 13 1.5 Grand Challenges of Computational Intelligence 14 1.6 Overview of the Book 14 1.7 Matlab Basics 16 1.8 Bibliography 17 Chapter 2: Fuzzy Logic 21-78 2.1 Introduction 21 2.2 Fuzzy Logic 23 2.3 Fuzzy Sets 24 2.4 Membership Functions 25 2.5 Features of MFs 30 2.6 Operations on Fuzzy sets 32 2.7 Linguistic Variables 39 2.8 Linguistic Hedges 42 2.9 Fuzzy Relations 45 2.10 Fuzzy If-Then Rules 48 2.11 Fuzzification 52 2.12 Defuzzification 54 2.13 Inference Mechanism 59 2.13.1 Mamdani Fuzzy Inference 60 2.13.2 Sugeno Fuzzy Inference 61 2.13.3 Tsukamoto Fuzzy Inference 65 2.14 Worked out Examples 67 2.15 Matlab Programs 76 2.16 Bibliography 77 Chapter 3: Fuzzy Systems and Applications 79-128 3.1 Introduction 79 3.2 Fuzzy System 80 3.3 Fuzzy Modelling 81 3.3.1 Structure Identification 82 3.3.2 Parameter Identification 85 3.3.3 Construction of parameterised Membership Functions 86 3.4 Fuzzy Control 92 3.4.1 Fuzzification 93 3.4.2 Inference Mechanism 93 3.4.3 Rule-base 98 3.4.4 Defuzzification 100 3.5 Design of Fuzzy Controller 101 3.5.1 Input-output Selection 102 3.5.2 Choice of Membership Functions 102 3.5.3 Creation of Rule-base 103 3.5.4 Types of Fuzzy Controller 104 3.6 Modular Fuzzy Controller .121 3.7 Matlab Programs 124 3.8 Bibliography 125 Chapter 4: Neural Networks 129-201 4.1 Introduction 129 4.2 Artificial Neuron Model 130 4.3 Activation Functions 132 4.4 Network Architecture 134 4.4.1 Feedforward Networks 134 4.4.1.1 Multilayer Perceptron (MLP) Networks 136 4.4.1.2 Radial Basis Function (RBF) Networks 138 4.4.1.3 General Regression Neural Networks 142 4.4.1.4 Probabilistic Neural Network 146 4.4.1.5 Belief Network 149 4.4.1.6 Hamming Network 150 4.4.1.7 Stochastic Networks 153 4.5 Learning in Neural Networks 153 4.5.1 Supervised learning 154 4.5.1.1 Widro-Hoff Learning Algorithm 155 4.5.1.2 Gradient Descent Rule 4.5.1.3 Generalised Delta Learning Rule 162 4.5.1.4 Backpropagation Learning Algorithm 165 4.5.1.5 Cohen-Grossberg Learning Rule 171 4.5.1.6 Adaptive Conjugate Gradient Model of Adeli and Hung 173 4.5.2 Unsupervised Learning 173 4.5.2.1 Hebbian Learning Rule 174 4.5.2.2 Kohonen Learning 178 4.6 Recurrent Neural Networks 187 4.6.1 Elman Networks 189 4.6.2 Jordan Networks 192 4.6.3 Hopfield Networks 194 4.7 Matlab Programs 198 4.8 Bibliography 198 Chapter 5: Neural Systems 202-232 5.1 Introduction 200 5.2 System Identification and Control 201 5.2.1 System Description 201 5.2.2 System Identification 202 5.2.3 System Control ..203 5.3 Neural Networks for Control 205 5.3.1 System Identification 206 5.3.2 Neural Networks for Control Design 208 5.3.2.1 NN-based direct (or specialised learning) control 209 5.3.2.2 NN-based indirect control .210 5.3.2.3 Backpropagation-through time control 211 5.3.2.4 NN-based direct inverse control 212 5.3.2.5 Model Predictive Control 214 5.3.2.6 NN-based Adaptive Control 216 5.3.2.7 NARMA-L2 (Feedback Linearization) Control 223 5.4 Matlab Programs 226 5.5 Bibliography 227 Chapter 6: Evolutionary Computation 233-304 6.1 Introduction 233 6.2 Evolutionary Computing 234 6.3 Terminologies of Evolutionary Computing 235 6.3.1 Chromosome Representation 235 6.3.2 Encoding Scheme 236 6.3.3 Population 243 6.3.4 Evaluation (or Fitness) Functions 245 6.3.5 Fitness Scaling 246 6.4 Genetic Operators 247 6.4.1 Selection Operators 247 6.4.2 Crossover Operators 252 6.4.3 Mutation Operators 261 6.5 Performance Measure of EA 264 6.6 Evolutionary Algorithms 265 6.6.1 Evolutionary Programming 265 6.6.2 Evolution Strategies 271 6.6.3 Genetic Algorithms 277 6.6.4 Genetic Programming 283 6.6.5 Differential Evolution 294 6.6.6 Cultural Algorithm 299 6.7 Matlab Programs 300 6.8 Bibliography 301 Chapter 7: Evolutionary Systems 305-340 7.1 Optimisation .305 7.2 Multi-objective Optimisation ..310 7.2.1 Vector Evaluated GA 315 7.2.2 Multi-objective GA 315 7.2.3 Niched Pareto GA .316 7.2.4 Non-dominated Sorting GA 316 7.2.5 Strength Pareto Evolutionary Algorithm 318 7.3 Co-evolution .319 7.3.1 Cooperative Co-evolution 324 7.3.2 Competitive Co-evolution .326 7.4 Parallel Evolutionary Algorithms 328 7.4.1 Global GA 329 7.4.2 Migration (or Island) Model GA 330 7.4.3 Diffusion GA .331 7.4.4 Hybrid Parallel GA 334 7.5 Bibliography .336 Chapter 8: Evolutionary Fuzzy Systems 341-392 8.1 Introduction 341 8.2 Evolutionary Adaptive Fuzzy Systems 343 8.2.1 Evolutionary Tuning of Fuzzy Systems 345 8.2.2 Evolutionary Learning of Fuzzy Systems 361 8.3 Objective Functions and Evaluation 368 8.3.1 Objective Functions 368 8.3.2 Evaluation 370 8.4 Fuzzy Adaptive Evolutionary Algorithms 371 8.4.1 Fuzzy Logic based Control of EA Parameters 374 8.4.2 Fuzzy Logic based Genetic Operators of EA 387 8.5 Bibliography 388 Chapter 9: Evolutionary Neural Systems 393-455 9.1 Introduction 393 9.2 Supportive Combinations 395 9.2.1 NN-EA Supportive Combination 395 9.2.2 EA-NN Supportive Combination 398 9.3 Collaborative Combinations 406 9.3.1 EA for NN Connection Weight Training 408 9.3.2 EA for NN Architectures 416 9.3.3 EA for NN Node Transfer Functions 430 9.3.4 EA for NN Weight, Architecture and Transfer Function Training 434 9.4 Amalgamated Combination 437 9.5 Competing Conventions 440 9.6 Bibliography 447 Chapter 10: Neuro Fuzzy Systems 455-530 10.1 Introduction 455 10.2 Combination of Neural and Fuzzy Systems 458 10.3 Cooperative Neuro-Fuzzy Systems 459 10.3.1 Cooperative FS-NN Systems 460 10.3.2 Cooperative NN-FS Systems 461 10.4 Concurrent Neuro-Fuzzy Systems 470 10.5 Hybrid Neuro-Fuzzy Systems 471 10.5.1 Fuzzy Neural Networks with Mamdani-type Fuzzy Inference System 472 10.5.2 Fuzzy Neural Networks with Takagi-Sugeno-type Fuzzy Inference System 474 10.5.3 Fuzzy Neural Networks with Tsukamoto-type Fuzzy Inference System 476 10.5.4 Neural Network based Fuzzy System (Sigma-Pi Network) 480 10.5.5 Fuzzy-Neural System Architecture with Ellipsoid Input Space 484 10.5.6 Fuzzy Adaptive Learning Control Network (FALCON) 487 10.5.7 Approximate Reasoning based Intelligent Control (ARIC) 490 10.5.8 Generalised ARIC (GARIC) 495 10.5.9 Fuzzy Basis Function Networks (FBFN) 502 10.5.10 FUzzy Net (FUN) 505 10.5.11 Combination of Fuzzy Inference and Neural Network in Fuzzy Inference Software (FINEST) 507 10.5.12 Neuro-Fuzzy Controller (NEFCON) 510 10.5.13 Self-constructing Neural Fuzzy Inference Network (SONFIN) 512 10.6 Adaptive Neuro-Fuzzy System 515 10.6.1 Adaptive Neuro-Fuzzy Inference System (ANFIS) 516 10.6.2 Coactive Neuro-Fuzzy Inference System (CANFIS) 519 10.7 Fuzzy Neurons 523 10.8 Matlab Programs 526 10.9 Bibliography 527 Appendix531-606 Index