Advances in Feature Selection for Data and Pattern Recognition (eBook)

Preface 6
Contents 8
Editors and Contributors 15
1 Advances in Feature Selection for Data and Pattern Recognition: An Introduction 19
1.1 Introduction 19
1.2 Chapters of the Book 21
1.3 Concluding Remarks 25
References 25
Part I Nature and Representation of Data 28
2 Attribute Selection Based on Reduction of Numerical Attributes During Discretization 29
2.1 Introduction 29
2.2 Dominant Attribute Discretization 30
2.3 Multiple Scanning Discretization 35
2.4 Experiments 35
2.5 Conclusions 39
References 39
3 Improving Bagging Ensembles for Class Imbalanced Data by Active Learning 41
3.1 Introduction 41
3.2 Improving Classifiers Learned from Imbalanced Data 44
3.2.1 Nature of Imbalanced Data 44
3.2.2 Evaluation of Classifiers on Imbalanced Data 45
3.2.3 Main Approaches to Improve Classifiers for Imbalanced Data 47
3.3 Active Learning 48
3.4 Ensembles Specialized for Imbalanced Data 49
3.5 Active Selection of Examples in Under-Sampling Bagging 51
3.6 Experimental Evaluation 55
3.6.1 Experimental Setup 55
3.6.2 Results of Experiments 56
3.7 Conclusions 64
References 66
4 Attribute-Based Decision Graphs and Their Roles in Machine Learning Related Tasks 69
4.1 Introduction 70
4.2 The Attribute-Based Decision Graph Structure (AbDG) for Representing Training Data 72
4.2.1 Constructing an AbDG 72
4.2.2 Assigning Weights to Vertices and Edges 74
4.2.3 Computational Complexity for Building an AbDG 77
4.3 Using the AbDG Structure for Classification Tasks 78
4.4 Using an AbDG for Classification Purposes - A Case Study 79
4.5 Using the AbDG Structure for Imputation Tasks 80
4.6 Searching for Refined AbDG Structures via Genetic Algorithms 82
4.7 Conclusions 85
References 86
5 Optimization of Decision Rules Relative to Length Based on Modified Dynamic Programming Approach 88
5.1 Introduction 88
5.2 Background 90
5.3 Main Notions 91
5.4 Modifed Algorithm for Directed Acyclic Graph Construction ??ast(T) 93
5.5 Procedure of Optimization Relative to Length 96
5.6 Experimental Results 98
5.6.1 Attributes' Values Selection and Size of the Graph 98
5.6.2 Comparison of Length of ?-Decision Rules 100
5.6.3 Classifier Based on Rules Optimized Relative to Length 105
5.7 Conclusions 106
References 106
Part II Ranking and Exploration of Features 109
6 Generational Feature Elimination and Some Other Ranking Feature Selection Methods 110
6.1 Introduction 111
6.2 Selected Methods and Algorithms 112
6.2.1 Rough Set Based Feature Selection 113
6.2.2 Random Forest Based Feature Selection 115
6.2.3 Generational Feature Elimination Algorithm 116
6.3 Feature Selection Experiments 118
6.4 Results and Conclusions 120
References 123
7 Ranking-Based Rule Classifier Optimisation 126
7.1 Introduction 126
7.2 Background 127
7.2.1 Attribute Ranking 128
7.2.2 Rule Classifiers 129
7.2.3 Filtering Rules 130
7.3 Research Framework 131
7.3.1 Preparation of the Input Datasets 131
7.3.2 Rankings of Features 132
7.3.3 DRSA Decision Rules 133
7.3.4 Weighting Rules 135
7.4 Experimental Results 136
7.4.1 Filtering Rules by Attributes 136
7.4.2 Filtering Rules by Weights 139
7.4.3 Summary of Results 140
7.5 Conclusions 142
References 143
8 Attribute Selection in a Dispersed Decision-Making System 145
8.1 Introduction 145
8.2 Related Works 146
8.3 Basics of the Rough Set Theory 147
8.4 An Overview of Dispersed Systems 148
8.4.1 Basic Definitions 149
8.4.2 Static Structure 150
8.4.3 Dynamic Structure with Disjoint Clusters 151
8.4.4 Dynamic Structure with Inseparable Clusters 153
8.4.5 Dynamic Structure with Negotiations 153
8.5 Description of the Experiments 155
8.5.1 Data Sets 155
8.5.2 Attribute Selection 156
8.5.3 Evaluation Measures and Parameters Optimization 157
8.6 Experiments and Discussion 160
8.6.1 Results for Static Structure 160
8.6.2 Results for Dynamic Structure with Disjoint Clusters 169
8.6.3 Results for Dynamic Structure with Inseparable Clusters 169
8.6.4 Results for Dynamic Structure with Negotiations 169
8.6.5 Comparison of All Methods 170
8.6.6 Comparison for Data Sets 172
8.7 Conclusion 172
References 173
9 Feature Selection Approach for Rule-Based Knowledge Bases 175
9.1 Introduction 175
9.2 Feature Selection Methods for Rule Mining Processes 176
9.2.1 Significance of Rules Mining Process 178
9.2.2 Feature Selection in the Rules and Their Clusters 179
9.2.3 Related Works 180
9.2.4 Clustering the Rules Based on the Similarity Approach 180
9.2.5 Clustering the Rules 182
9.2.6 Cluster's Representative 184
9.3 Rough Set Approach in Creating Rules' Clusters Representatives 185
9.3.1 Lower and Upper Approximation Approach 186
9.3.2 KbExplorer - A Tool for Knowledge Base Exploration 187
9.4 Experiments 188
9.4.1 From Original Data to the Rules 188
9.4.2 The Results of the Experiments 188
9.4.3 The Summary of the Experiments 192
9.5 Conclusion 192
References 193
Part III Image, Shape, Motion, and Audio Detection and Recognition 195
10 Feature Selection with a Genetic Algorithm for Classification of Brain Imaging Data 196
10.1 Introduction 197
10.2 Materials and Methods 200
10.2.1 Data and Preprocessing 200
10.2.2 Least Absolute Shrinkage and Selection Operator 200
10.2.3 Minimalist Genetic Algorithm for Feature Selection 202
10.2.4 Experimental Set-Up 205
10.3 Results 206
10.4 Discussion 209
10.5 Conclusions 210
References 211
11 Shape Descriptions and Classes of Shapes. A Proximal Physical Geometry Approach 214
11.1 Introduction 215
11.2 Introduction to Simplicial Complexes 216
11.2.1 Examples: Detecting Shapes from Simplicial Complexes 217
11.3 Preliminaries of Proximal Physical Geometry 220
11.3.1 Image Object Shape Geometry 220
11.3.2 Descriptions and Proximities 221
11.3.3 Descriptive Proximities 223
11.3.4 Edelsbrunner--Harer Nerve Simplexes 226
11.4 Features of Image Object Shapes 229
11.5 Concluding Remarks 234
References 234
12 Comparison of Classification Methods for EEG Signals of Real and Imaginary Motion 237
12.1 Introduction 238
12.2 EEG Signal Parameterisation 239
12.3 Data Classification Method 242
12.4 Classification Results 244
12.5 Conclusions 246
References 247
13 Application of Tolerance Near Sets to Audio Signal Classification 250
13.1 Introduction 250
13.2 Related Work 252
13.3 Automatic Detection of Video Blocks 254
13.4 Audio and Video Features 255
13.4.1 Audio Features 255
13.4.2 Video Features 258
13.5 Theoretical Framework: Near Sets and Tolerance Near Sets 258
13.5.1 Preliminaries 259
13.5.2 Tolerance Near Sets 261
13.6 Tolerance Class Learner - TCL 263
13.6.1 Algorithms 264
13.6.2 Phase II: Classification 266
13.7 Experiments 266
13.7.1 Speech and Non-speech Dataset 266
13.7.2 Discussion 268
13.7.3 Music and Music + Speech Dataset 269
13.7.4 Detection of Commercial Blocks in News Data 270
13.8 Conclusion and Future Work 272
References 272
Part IV Decision Support Systems 276
14 Visual Analysis of Relevant Features in Customer Loyalty Improvement Recommendation 277
14.1 Introduction 278
14.2 Related Applications 278
14.3 Dataset and Application 279
14.3.1 Dataset 279
14.4 Decision Problem 280
14.4.1 Attribute Analysis 280
14.4.2 Attribute Reduction 281
14.4.3 Customer Satisfaction Analysis and Recognition 281
14.4.4 Providing Recommendations 282
14.5 Proposed Approach 282
14.5.1 Machine Learning Techniques 282
14.5.2 Visualization Techniques 290
14.6 Evaluation Results 294
14.6.1 Single Client Data (Local) Analysis 295
14.6.2 Global Customer Sentiment Analysis and Prediction 296
14.6.3 Recommendations for a Single Client 297
14.7 Conclusions 300
References 300
15 Evolutionary and Aggressive Sampling for Pattern Revelation and Precognition in Building Energy Managing System with Nature-Based Methods for Energy Optimization 302
15.1 Introduction 303
15.2 Ant Colony Optimization 303
15.2.1 Ant System 303
15.2.2 Ant Colony Optimization Metaheuristics 305
15.2.3 Ant Colony System 306
15.2.4 Type of Issues That Are Able To Be Solved Using ACO 309
15.3 Building's Installation as 3D Matrix-Like Grids 310
15.4 Classical ACO Approach in Revealing a Pattern 313
15.4.1 Capability of Revealing with ACO 313
15.5 Aggressive ACO Sampling in Revealing a Pattern 315
15.5.1 Ant Decomposition and Specialization 315
15.5.2 Precognition Paths 316
15.5.3 Ants Spreading 318
15.6 Algorithm and Procedures 319
15.7 Conclusion 324
References 325
Index 327