Computational Intelligence (eBook)

Title Page 2
Preface 8
Contents 11
Part I Introduction 14
Synergy in Computational Intelligence 15
Introduction 15
The Birth of Computational Intelligence 16
The Main CI Techniques 18
Evolutionary Algorithms 18
Neural Networks 20
Fuzzy Systems 21
Multi-Agent Systems 23
Other Techniques Covered in the Book 24
Chapters Included in This Book 24
Part I: Introduction 25
Part II: Fusing Evolutionary Algorithms and Fuzzy Logic 25
Part III: Adaptive Solution Schemes 26
Part IV: Multi-Agent Systems 27
Part V: Computer Vision 28
Part VI: Communication for CI Systems 29
Part VII: Artificial Immune Systems 30
Part VIII: Parallel Evolutionary Algorithms 30
Part IX: CI for Clustering and Classification 31
References 33
Computational Intelligence: The Legacy of Alan Turing and John von Neumann 34
Introduction 34
Turing and Machine Intelligence 36
Turing’s Construction of an Intelligent Machine 37
Turing on Learning and Evolution 38
Turing and Neural Networks 39
Discipline and Initiative 40
Von Neumann’s Logical Theory of Automata 41
McCulloch-Pitts Theory of Formal Neural Networks 42
Complication and Self-reproduction 43
Holland’s Logical Theory of Adaptive Systems 44
The Beginning of Artificial Intelligence - The Logic Theorist 46
Discussion of the Early Proposals to Create Artificial Intelligence by Simulating Evolution 47
Cyc and Cog: Two Large Projects in the Legacy of Alan Turing 49
The Cyc Project 49
The Cog Project 50
The JANUS Hand-Eye Robot and the Pandemonium Architecture 51
Conclusion 52
References 53
Part II Fusing Evolutionary Algorithms and Fuzzy Logic 55
Multiobjective Evolutionary Algorithms for Electric Power Dispatch Problem 56
Introduction 56
EED Problem Formulation 59
Problem Objectives 59
Problem Constraints 60
Problem Formulation 61
Multiobjective Optimization 61
Principles and Definitions 61
Fitness Assignment 62
Diversity Preservation 63
Multiobjective Evolutionary Algorithms 63
Non-dominated Sorted Genetic Algorithm (NSGA) 63
Niched Pareto Genetic Algorithm (NPGA) 65
Strength Pareto Evolutionary Algorithm (SPEA) 67
MOEA Implementation 68
Reducing the Pareto Set by Clustering 68
Best Compromise Solution 69
Real-Coded Genetic Algorithm 70
The Computational Flow 71
Settings of the Proposed Approach 71
Results and Discussions 74
A Comparative Study 82
Future Work 88
Conclusions 88
References 89
Fuzzy Evolutionary Algorithms and Genetic Fuzzy Systems: A Positive Collaboration between Evolutionary Algorithms and Fuzzy Systems 92
Introduction 92
Fuzzy Rule Based Systems 93
Preliminaries: Fuzzy Set and Linguistic Variable 94
Basic Elements of FRBSs 96
Example of FRBS: Fuzzy Logic Control of an Inverted Pendulum 98
Genetic Fuzzy Systems 100
Taxonomy of Genetic Fuzzy Systems 101
Genetic Learning: Rule Coding and Cooperation/Competition Evolutionary Process 106
Some GFS Milestones: Books and Special Issues 107
Current Research Trends in GFSs 110
Fuzzy Evolutionary Algorithms 114
Fuzzy Adaptive GAs 114
EA Components Based on Fuzzy Tools 119
Other Fuzzy EA Models 122
Future Work on Fuzzy EAs 125
Concluding Remarks 128
References 128
Multiobjective Genetic Fuzzy Systems 140
Introduction 140
Evolutionary Multiobjective Optimization 145
Some Basic Concepts in Multiobjective Optimization 145
Evolutionary Multiobjective Optimization 146
Multiobjective Genetic Fuzzy Systems 149
Multiobjective Genetic Fuzzy Rule Selection 151
Fuzzy Rule-Based Classifiers 151
Candidate Rule Generation 153
Multiobjective Fuzzy Rule Selection 154
Computational Experiments 155
Multiobjective Fuzzy Genetics-Based Machine Learning 163
Two Approaches in Genetics-Based Machine Learning 163
Implementation of Multiobjective Fuzzy GBML Algorithms 163
Computational Experiments 166
Related Studies 170
Evolutionary Multiobjective Data Mining 170
Evolutionary Multiobjective Feature Selection 171
Evolutionary Multiobjective Clustering 171
Evolutionary Ensemble Design 172
Evolutionary Multiobjective Neural Network Design 172
Multiobjective Genetic Programming 172
Future Research Directions 172
Conclusions 174
References 174
Part III Adaptive Solution Schemes 183
Exploring Hyper-heuristic Methodologies with Genetic Programming 184
Introduction 184
The Need for Heuristics 185
Hyper-heuristics 185
Genetic Programming 186
Chapter Outline 189
Genetic Programming as a Hyper-heuristic 189
Suitability of Genetic Programming as a Hyper-heuristic 189
The Basic Approach 190
Case Studies 190
Boolean Satisfiability – SAT 191
Online Bin Packing 194
Literature Review 199
Genetic Programming Hyper-heuristics for Generating Reusable Heuristics 199
Genetic Programming Hyper-heuristics for Generating Disposable Heuristics 202
Learning to Learn 202
The Teacher System 203
Related Areas 203
Summary and Conclusion 204
The Need for Automatic Heuristic Generation 204
Supplementing Human Designed Heuristics 204
References 205
Adaptive Constraint Satisfaction: The Quickest First Principle 209
Introduction 209
The Adaptive Strategy 211
Chain Design 211
Switching Policy 212
The Reduced Exceptional Behaviour Algorithm (REBA) 213
The REBA Algorithm Chain 213
The Monitor Search Level (MSL) Thrashing Predictor 214
The REBA Switching Mechanism 218
Experiments 218
Experimental Design 218
The Effectiveness of REBA 220
Evaluation of the MSL Predictor 222
Discussion 225
Appendix 227
A.1 Tables of results for Figures 4–7 227
A.2 Results for 100 Variables 229
A.3 Tables of results for Figures 14 – 17 231
References 235
Part IV Multi-Agent Systems 237
Collaborative Computational Intelligence in Economics 238
Introduction 238
Heterogeneous Agents 239
The Three Levels of Collaboration 239
Macroscopic Level: Evolving Population 240
Microscopic Level: Heterogeneity in Intelligence 250
Molecule Level: Hybridization 258
Human and Software Agents 259
Mirroring 260
Competition 262
Collaboration 264
Hybrid Systems 265
Nature of Hybridization 265
Evolutionary-Based Hybridization 266
Semantics-Based Hybrid Systems 269
Feature Reduction: Rough GA or GP 270
Concluding Remarks 271
References 272
IMMUNE: A Collaborating Environment for Complex System Design 279
Introduction 280
Design Planning for Complex Products Development 282
The Necessity of Emulating Low Level Product Knowledge for Complex Product Design Planning 284
Concurrent Parametric Design 285
Simulation Based Engineering, and Complexity Measures 286
Deriving a Team Based DSM from a Simulated Parameter Based DSM 289
Radical Innovation 292
Holistic Process Monitoring 294
Artificial Immune Systems (AIS) 297
IMMUNE: A Collaborating Architecture 299
Blackboard Architecture 303
Control Source 303
Agents Structures 306
Implementation and Overall Behavior 309
Structuration: Adaptive Organization Structure with Virtual Cross-Functional Teams 310
Global Decomposition Strategies and Modality 314
Conclusion 318
References 320
Bayesian Learning for Cooperation in Multi-Agent Systems 325
Introduction 325
Background 327
The Disaster Response Domain 327
Single Agent Decision Making 328
Coordinated Decision Making 332
Bayesian Learning Models 336
Bayesian Learning 336
MDPs and POMDPs 337
Bayesian Learning Approximation Using Finite State Machines 341
Definitions 341
Learning FSMs 342
Online Solutions: Best Response 344
An Online Learning Algorithm 345
Model Instantiation 347
Experimental Evaluation 349
Experimental Setup 350
Examining the Learning Rate 350
Varying the Sampling Rate 352
Varying the Visibility 354
Varying the Victim Arrival Rate 356
Varying Scaling Factors 356
Conclusions and Future Work 360
References 362
Collaborative Agents for Complex Problems Solving 365
Introduction 365
Self-interested and Cooperative Multi-Agent Systems 366
Traditional Classification 366
The Blurred Boundary 367
Two Scenarios 367
Collaborative Problem Solving through Agent Cooperation 368
Agent Cooperation in Agent Teams: The Scenario 370
One-Shot and Long-Term Team-Formation Mechanisms 373
Flexible Team-Formation Mechanism 375
Experiments 381
Summary 384
Collaborative Problem Solving through Agent Competition 384
Traditional Agent Negotiation 384
Partner Selection in Agent Negotiation 388
Behavior Prediction in Agent Negotiation 392
Conclusion 400
References 401
Part V Computer Vision 404
Predicting Trait Impressions of Faces Using Classifier Ensembles 405
Introduction 405
Face Classification: Single Classifier Algorithms and Collaborative Methods 410
Single Classifier Algorithms 415
Classifier Ensembles 422
Resources 424
Study Design 424
Step 1: Generation of Stimulus Faces 425
Step 2: Assessing Trait Impressions of Stimulus Faces 426
Step 3: Division of Stimulus Faces into Trait Class Sets 427
Classification Experiments 429
System Architecture 429
Results 432
Conclusion 435
Appendix 437
References 438
The Analysis of Crowd Dynamics: From Observations to Modelling 442
Introduction 442
Background 444
Crowd Information Extraction 446
Crowd Modelling and Events Inference 449
Examples of Bridging the Research 450
Measuring Crowd Motion 451
Method 1: Pyramid-Based Interest Points Topological Matching 451
Method 2: Using Edge Continuity Constrains of Interest Points 453
Comparison of the Two Methods 455
Testing Based on Motion Connect Component 457
Modelling Crowd Dynamics 461
Statistical Analysis 461
Path Discovery 462
Self-Organizing Map for Learning Crowd Dynamics 463
Discussion 467
References 468
Part VI Communications for CI Systems 474
Computational Intelligence for the Collaborative Identification of Distributed Systems 475
Introduction 475
Sensor Networks: The State of the Art 476
Identification of Distributed Systems 478
Modeling a Distributed System by the Karhunen-Lo/'{e}ve Transform 480
Identification of a Distributed System Knowing the Output y 482
Neural Network Based Identification 483
Identification of a Distributed System by a Network of Independent Sensors 485
Two Sensors (S = 2) 486
S Sensors 487
Best Estimate of the Matrix $/Psi$ Based on the Distributed KLT Algorithm 489
Experimental Results 490
First Experiment: Parabolic PDE 490
Second Experiment: Hyperbolic PDE 491
Conclusions 498
References 498
Collaboration at the Basis of Sharing Focused Information: The Opportunistic Networks 501
Introduction 501
The Opportunistic Networks Framework 503
A Case Study to Lead the Theory 505
An Elementary Mathematical Model 507
A Very General Way of Maintaining Memory in a Time Process 509
The Timing of the Intentional Process 511
Validating the Model 513
Drawing Data and Models from the Literature 513
A Homemade Validation 514
The Architecture 515
Preliminary Matches 516
The Statistical Versant 518
Exploiting the Model 520
Conclusions 521
References 522
Part VII Artificial Immune Systems 525
Exploiting Collaborations in the Immune System: The Future of Artificial Immune Systems 526
Introduction 526
A Reflection on AIS Today 527
Challenges Posed by Real Systems 529
The Natural Immune System 531
Innate vs. Adaptive Immunology 532
Cooperative Innate Immunology 532
Dendritic Cells 534
The Adaptive Immune System: Carneiro’s Networks 536
The Cognitive Immune System 541
Interpreting Immune Collaborations in Real-World Applications 544
Application of Carneiro’s Model in a Machine Learning Scenario 544
The Relationship of the Carneiro Model with Theoretical Machine Learning 546
A Practical Perspective: Application of Innate and Adaptive Immune Mechanisms to WSN 547
Immune Approaches to SpeckNets 549
The Future of AIS: Immuno-Engineering 552
Conclusions 553
References 554
Part VIII Parallel Evolutionary Algorithms 558
Evolutionary Computation: Centralized, Parallel or Collaborative 559
Introduction 559
Darwinism - The Unfinished Theory 560
The System View of Evolution 563
Evolutionary Algorithms - Centralized, Parallel or Collaborative 565
Co-evolution and Collaboration in Evolution 567
Darwin Revisited 567
Spatial Population Structures in Evolution Theories 568
The Iterated Prisoner’S Dilemma as an Evolutionary Game 570
The Simulation of Spatial Structures Using the Iterated Prisoner’s Dilemma 571
The Genetic Representation 572
Mathematical Analysis of Structured Populations in Evolutionary Games 573
Simulation Results 574
The Punctuated Equilibrium Theory 579
Combinatorial Optimization by the PGA 579
The Traveling Salesman Problem 580
The Graph Partitioning Problem 582
Continuous Function Optimization by Competition 586
The BGA for Continuous Parameter Optimization 586
Competition between Subpopulations 587
The Basic Competition Model of the BGA 588
The Extended Competition Model 589
Conclusion 590
References 591
Part IX CI for Clustering and Classification 594
Fuzzy Clustering of Likelihood Curves for Finding Interesting Patterns in Expression Profiles 595
Introduction 595
Background to Quantitative Proteomics and iTRAQ$^{TM}$ Based Likelihood Curves 597
Proteomics 598
Identification and Characterisation of Proteins Based on LC-MS 598
Quantification of Peptides and Proteins 599
Impreciseness of Regulatory Information: Intensity-Dependent Noise 602
Calculation and Visualisation of Regulatory Information 605
Fuzzy Cluster Analysis 608
Fuzzy Clustering of Likelihood Curves 610
Generation of Prototypes 611
Validity Measures 613
Examples 614
Conclusions 617
References 618
A Hybrid Rule-Induction/Likelihood-Ratio Based Approach for Predicting Protein-Protein Interactions 619
Introduction 619
Computational Prediction of Protein-Protein Interactions 620
Overview of the Proposed Method 621
Organisation 622
Classification Rule Discovery Algorithms 622
Separate-and-Conquer Approach 623
Divide-and-Conquer Approach 623
Protein Interaction Data and Predictive Features 625
A New Hybrid Rule Induction/Likelihood-Ratio Based Method 626
From Naive Bayes to a Likelihood Based Approach for the Prediction of Protein-Protein Interactions 626
Generating Classification Rules for Protein-Protein Interaction Prediction 627
Classification Rule Discovery as a Binning Method for a Likelihood-based Approach 628
Results and Discussion 628
Conclusions 630
References 631
Improvements in Flock-Based Collaborative Clustering Algorithms 634
Introduction 634
Swarm Intelligence Clustering 636
Particle Swarm Clustering 637
Ant Clustering 638
Flocks of Agents for Data Visualization and Clustering 640
Flocks of Agents Based-Data Visualization 641
Flocks of Agents-Based Clustering 641
Improved Distance Threshold Estimates 646
Alternative Fixed Thresholding 646
Adaptive Thresholding Using FClust-Annealing 646
The (K-means/FClust) Hybrid Algorithm 647
K-Means Algorithm 647
(K-means+FClust) Hybrid 648
Stopping Criterion 649
Experimental Results 650
Datasets 650
Post Processing 651
Results 652
Conclusions and Future Work 664
References 665
Combining Statistics and Case-Based Reasoning for Medical Research 668
Introduction 668
Case-Based Reasoning in Medicine 669
The ISOR Approach 670
Incremental Development of an Explanation Model for Exceptional Dialysis Patients 672
Setting up a Model 673
Setting up a Case Base 675
Another Problem 677
Example 678
Illustration of ISOR’s Program Flow 679
Missing Data 681
The Data Set 682
Restoration of Missing Data 682
Results 686
Experimental Results 686
Restoration of Real Missing Data and Setting up a New Model 688
Conclusion 689
References 690
Collaborative and Experience-Consistent Schemes of System Modelling in Computational Intelligence 692
Introductory Comments 693
Collaborative Clustering 695
The General Flow of Collaborative Processing 698
Algorithmic Aspects of Collaborative Clustering 699
The Computing Scheme 699
Evaluation of the Quality of Collaboration: Striking a Sound Compromise between Global and Local Characteristics of Data 700
Fuzzy Sets of Type-2 in the Quantification of the Effect of Collaboration 702
Collaborative Clustering in Presence of Different Levels of Information Granularity 703
Hierarchical Clusters of Clusters 705
Experience-Consistent Fuzzy Modeling 706
The Consistency-Based Optimization of Local Regression Models 708
The Alignment of Information Granules 712
Characterization of Experience-Consistent Models through its Granular Parameters 712
Experience-Consistent Design of Radial-Basis Function Neural Networks 714
Conclusions 717
References 717
Index 719