Principles of Soft Computing Using Python Programming

Gypsy Nandi, PhD, is an Assistant Professor in the Department of Computer Applications at Assam Don Bosco University, Guwahati, India, and Coordinator of VanitAgrata, an empowerment cell of Assam Don Bosco University that teaches rural women digital awareness skills. She has authored two books on data science and soft computing and has also published extensively on machine learning and related subjects.

About the Author xi

Preface xiii

1 Fundamentals of Soft Computing 1

1.1 Introduction to Soft Computing 1

1.2 Soft Computing versus Hard Computing 2

1.3 Characteristics of Soft Computing 4

1.4 Components of Soft Computing 7

1.4.1 Fuzzy Computing 7

1.4.2 Neural Network 10

1.4.3 Evolutionary Computing 12

1.4.4 Machine Learning 19

1.4.5 Other Techniques of Soft Computing 29

2 Fuzzy Computing 35

2.1 Fuzzy Sets 37

2.1.1 Features of Fuzzy Membership Functions 38

2.2 Fuzzy Set Operations 41

2.3 Fuzzy Set Properties 42

2.4 Binary Fuzzy Relation 45

2.5 Fuzzy Membership Functions 46

2.6 Methods of Membership Value Assignments 49

2.7 Fuzzification vs. Defuzzification 58

2.8 Fuzzy c-Means 62

3 Artificial Neural Network 75

3.1 Fundamentals of Artificial Neural Network (ANN) 76

3.2 Standard Activation Functions in Neural Networks 81

3.2.1 Binary Step Activation Function 81

3.2.2 Linear Activation Function 83

3.2.3 Sigmoid/Logistic Activation Function 84

3.2.4 ReLU Activation Function 85

3.2.5 Tanh Activation Function 87

3.2.6 Leaky ReLU Activation Function 88

3.2.7 SoftMax Activation Function 90

3.3 Basic Learning Rules in ANN 91

3.3.1 Hebbian Learning Rule 93

3.3.2 Perceptron Learning Rule 93

3.3.3 Delta Learning Rule 94

3.3.4 Correlation Learning Rule 94

3.3.5 Competitive Learning Rule 95

3.3.6 Outstar Learning Rule 95

3.4 McCulloch–Pitts ANN Model 96

3.5 Feed-Forward Neural Network 98

3.5.1 Single-Layer Perceptron 99

3.5.2 Multilayer Perceptron 103

3.5.3 Radial Basis Function Network 107

3.6 Feedback Neural Network 111

3.6.1 Self-Organizing Map (SOM) 111

3.6.2 Hopfield Neural Network (HNN) 116

4 Deep Learning 123

4.1 Introduction to Deep Learning 123

4.2 Classification of Deep Learning Techniques 125

4.2.1 Convolutional Neural Networks 125

4.2.2 Recurrent Neural Network (RNN) 137

4.2.3 Generative Adversarial Network (GAN) 144

4.2.4 Autoencoders 149

5 Probabilistic Reasoning 159

5.1 Introduction to Probabilistic Reasoning 159

5.1.1 Random Experiment 160

5.1.2 Random Variables 160

5.1.3 Independence 161

5.1.4 Sample Space 162

5.1.5 Odds and Risks 162

5.1.6 Expected Values 165

5.2 Four Perspectives on Probability 165

5.2.1 The Classical Approach 166

5.2.2 The Empirical Approach 166

5.2.3 The Subjective Approach 167

5.2.4 The Axiomatic Approach 167

5.3 The Principles of Bayesian Inference 168

5.4 Belief Network and Markovian Network 171

5.4.1 Syntax and Semantics 172

5.4.1.1 Belief Network 172

5.4.1.2 Markovian Network 172

5.4.2 Conditional Independence 172

5.4.3 Learning Methods of the Networks 177

5.5 Hidden Markov Model 178

5.6 Markov Decision Processes 186

5.7 Machine Learning and Probabilistic Models 191

6 Population-Based Algorithms 197

6.1 Introduction to Genetic Algorithms 197

6.2 Five Phases of Genetic Algorithms 198

6.2.1 Population Initialization 198

6.2.2 Fitness Function Calculation (Evaluation) 199

6.2.3 Parent Selection 200

6.2.4 Crossover 204

6.2.5 Mutation 205

6.3 How Genetic AlgorithmWorks? 207

6.4 Application Areas of Genetic Algorithms 212

6.4.1 Using GA in Travelling Salesman Problem 212

6.4.2 Using GA in Vehicle Routing Problem 216

6.5 Python Code for Implementing a Simple Genetic Algorithm 221

6.6 Introduction to Swarm Intelligence 225

6.7 Few Important Aspects of Swarm Intelligence 227

6.7.1 Collective Sorting 228

6.7.2 Foraging Behavior 228

6.7.3 Stigmergy 229

6.7.4 Division of Labor 229

6.7.5 Collective Transport 230

6.7.6 Self-Organization 231

6.8 Swarm Intelligence Techniques 233

6.8.1 Ant Colony Optimization 233

6.8.1.1 How ACO TechniqueWorks? 233

6.8.1.2 Applying ACO to Optimization Problems 235

6.8.1.3 Using ACO in Travelling Salesman Problem 236

6.8.1.4 Python Code for Implementing ACO in TSP 239

6.8.2 Particle Swarm Optimization 241

6.8.2.1 How PSO TechniqueWorks? 242

6.8.2.2 Applying PSO to Optimization Problems 243

6.8.2.3 Using PSO in Job-Shop Scheduling Problem 246

6.8.2.4 Python Code for Implementing PSO 247

7 Rough Set Theory 255

7.1 The Pawlak Rough Set Model 255

7.1.1 Basic Terms in Pawlak Rough Set Model 256

7.1.2 Measures of Rough Set Approximations 260

7.2 Using Rough Sets for Information System 262

7.3 Decision Rules and Decision Tables 263

7.3.1 Parameters of Decision Tables 264

7.3.1.1 Consistency Factor 264

7.3.1.2 Support and Strength 265

7.3.1.3 Certainty Factor 265

7.3.1.4 Coverage Factor 266

7.3.2 Probabilistic Properties of Decision Tables 266

7.4 Application Areas of Rough Set Theory 268

7.4.1 Classification 268

7.4.2 Clustering 271

7.4.3 Medical Diagnosis 273

7.4.4 Image Processing 276

7.4.5 Speech Analysis 278

7.5 Using ROSE Tool for RST Operations 280

7.5.1 Attribute Discretization 280

7.5.2 Finding Lower and Upper Approximations 281

8 Hybrid Systems 289

8.1 Introduction to Hybrid Systems 289

8.2 Neurogenetic Systems 291

8.2.1 GA-based Weight Determination of Multilayer Feed-forward Net 292

8.2.2 Neuroevolution of Augmenting Topologies (NEAT) 298

8.3 Fuzzy-Neural Systems 302

8.3.1 Fuzzy Neurons 303

8.3.2 Adaptive Neuro-fuzzy Inference System (ANFIS) 307

8.4 Fuzzy-Genetic Systems 311

8.5 Hybrid Systems in Medical Devices 315

Exercises 322

Index 327