Data-Driven Prediction for Industrial Processes and Their Applications (eBook)

Jun Zhao is currently a Professor with the School of Control Science and Engineering, Dalian University of Technology, China. Chunyang Sheng is currently a lecturer with the School of Electrical Engineering and Automation, Shandong University of Science and Technology, China.  Wei Wang is currently a Professor with the School of Control Science and Engineering, Dalian University of Technology, China.

Preface 6
Audience and Goal of This Book 8
Acknowledgements 9
Contents 10
Chapter 1: Introduction 16
1.1 Why Prediction Is Required for Industrial Process 17
1.2 Category of Data-Based Industrial Process Prediction 18
1.2.1 Data Feature-Based Prediction 18
1.2.2 Time Scale-Based Prediction 18
1.2.3 Prediction Reliability-Based Prediction 19
1.3 Commonly Used Techniques for Industrial Prediction 20
1.3.1 Time Series Prediction Methods 20
1.3.2 Factor-Based Prediction Methods 21
1.3.3 Methods for PIs Construction 22
1.3.4 Long-Term Prediction Intervals Methods 23
1.4 Summary 24
References 25
Chapter 2: Data Preprocessing Techniques 27
2.1 Introduction 27
2.2 Anomaly Data Detection 29
2.2.1 K-Nearest-Neighbor 29
2.2.2 Fuzzy C Means 30
2.2.3 Adaptive Fuzzy C Means 32
2.2.4 Trend Anomaly Detection Based on AFCM and DTW 33
2.2.5 Deviants Detection Based on KNN-AFCM 36
2.2.6 Case Study 40
2.3 Data Imputation 43
2.3.1 Data-Missing Mechanism 43
2.3.2 Regression Filling Method 44
2.3.3 Expectation Maximum 44
2.3.4 Varied Window Similarity Measure 45
2.3.5 Segmented Shape-Representation Based Method 46
Key-Sliding-Window for Sequence Segmentation 46
Representation of Sequence Segmentation 48
Procedure of Data Imputation Based on Segmented Shape-Representation 50
2.3.6 Non-equal-Length Granules Correlation 51
Calculation for NGCC 53
NGCC-Based Correlation Analysis 55
Correlation-Based Data Imputation 55
2.3.7 Case Study 56
2.4 Data De-noising Techniques 60
2.4.1 Empirical Mode Decomposition 60
2.4.2 Case Study 61
2.5 Discussion 64
References 65
Chapter 3: Industrial Time Series Prediction 67
3.1 Introduction 67
3.2 Phase Space Reconstruction 69
3.2.1 Determination of Embedding Dimensionality 69
False Nearest-Neighbor Method (FNN) 70
Cao Method 70
3.2.2 Determination of Delay Time 71
Autocorrelation Function Method 71
Mutual Information Method 72
3.2.3 Simultaneous Determination of Embedding Dimensionality and Delay Time 72
3.3 Linear Models for Regression 74
3.3.1 Basic Linear Regression 74
3.3.2 Probabilistic Linear Regression 76
3.4 Gaussian Process-Based Prediction 78
3.4.1 Kernel-Based Regression 78
3.4.2 Gaussian Process for Prediction 80
3.4.3 Gaussian Process-Based ESN 81
3.4.4 Case Study 85
3.5 Artificial Neural Networks-Based Prediction 89
3.5.1 RNNs for Regression 89
3.5.2 ESN for Regression 93
3.5.3 SVD-Based ESN for Industrial Prediction 95
3.5.4 ESNs with Leaky Integrator Neurons 96
3.5.5 Dual Estimation-Based ESN 98
3.5.6 Case Study 101
Extended Kalman-Filter-Based Elman Network 102
SVD-Based ESN for Industrial Prediction 104
ESN with Leaky Integrator Neurons 106
Dual Estimation-Based ESN 110
3.6 Support Vector Machine-Based Prediction 113
3.6.1 Basic Concept of SVM 113
3.6.2 SVMs for Regression 115
3.6.3 Least Square Support Vector Machine 118
3.6.4 Sample Selection-Based Reduced SVM 119
3.6.5 Bayesian Treatment for LSSVM Regression 124
Probabilistic Interpretation of LSSVM Regressor (Level 1): Predictive Mean and Error Bars 124
Calculation of Maximum Posterior 124
Moderated Output of LSSVM Regressor 126
Inference of Hyper-Parameters (Level 2) 127
Inference of Kernel Parameters and Model Comparison 128
3.6.6 Case Study 128
3.7 Discussion 132
References 132
Chapter 4: Factor-Based Industrial Process Prediction 134
4.1 Introduction 134
4.2 Methods of Determining Factors 135
4.2.1 Gray Correlation 136
Case Study 137
4.2.2 Convolution-Based Methods 139
Case Study 140
4.2.3 Bayesian Technique of Automatic Relevance 141
4.3 Factor-Based Single-Output Model 144
4.3.1 Neural Networks-Based Model 144
4.3.2 T-S Fuzzy Model-Based Prediction 147
4.3.3 Multi-Kernels Least Square Support Vector Machine 148
4.3.4 Case Study 153
4.4 Factor-Based Multi-Output Model 156
4.4.1 Multi-Output Least Square Support Vector Machine 156
4.4.2 Case Study 159
4.5 Discussion 166
References 168
Chapter 5: Industrial Prediction Intervals with Data Uncertainty 171
5.1 Introduction 171
5.2 Commonly Used Techniques for Prediction Intervals 173
5.2.1 Delta Method 173
5.2.2 Mean and Variance-Based Estimation 175
5.2.3 Bayesian Method 177
5.2.4 Bootstrap Technique 179
5.3 Neural Networks-Based PIs Construction for Time Series 181
5.3.1 ESNs Ensemble-Based Prediction Model 181
5.3.2 Bayesian Estimation of the Uncertainties 182
5.3.3 Model Selection and Structural Optimization 185
5.3.4 Theoretical Analysis of the Prediction Performance 187
5.3.5 Case Study 188
Multiple Superimposed Oscillator 189
Application on Prediction for Generation Flow of BFG 193
5.4 Non-iterative NNs for PIs Construction 195
5.4.1 A Non-iterative Prediction Mode 196
5.4.2 Interval-Weighted ESN and Its Iterative Prediction 197
5.4.3 Gamma Test-Based Model Selection 199
5.4.4 Case Study 202
Two Benchmark Prediction Problems 202
Interval Prediction on the By-Product Gas System in Steel Industry 204
5.5 Gaussian Kernel-Based Causality PIs Construction 207
5.5.1 Mixed Gaussian Kernel for Regression 208
5.5.2 Mixed Gaussian Kernel for PIs Construction 209
5.5.3 Estimation of Effective Noise-Based Hyper-Parameters 211
5.5.4 Case Study 212
5.6 Prediction Intervals Construction with Noisy Inputs 214
5.6.1 Bayesian Estimation of the Output Uncertainty 215
5.6.2 Estimation of the External Input Uncertainties 217
5.6.3 Estimation of the Output Feedback Uncertainties 219
5.6.4 Estimation of the Total Uncertainties and PIs Construction 221
5.6.5 Case Study 223
5.7 Prediction Intervals with Missing Input 225
5.7.1 Kernel-Based DBN Prediction Model 225
5.7.2 Approximate Inference and PI Construction 226
5.7.3 Learning a Kernel-Based DBN 228
5.7.4 Case Study 229
5.8 Discussion 231
References 232
Chapter 6: Granular Computing-Based Long-Term Prediction Intervals 235
6.1 Introduction 235
6.2 Techniques of Granularity Partition 236
6.2.1 Partition of Equal Length 237
6.2.2 Partition of Unequal Length 238
Standard Granule Selection 241
Time Warping Normalization 241
6.3 Long-Term Prediction Model 243
6.3.1 Granular Model for Time Series Prediction 243
Case Study 245
BFG System Experiments 245
COG System Experiments 248
LDG System Experiments 250
6.3.2 Granular Model for Factor-Based Prediction 253
Case Study 259
Comparing with the Single-Output Model 259
Comparing with the Multi-output Model (Iteration Mechanism,) 261
6.4 Granular-Based Prediction Intervals 263
6.4.1 Initial PIs Construction 263
6.4.2 PIs Optimization 264
6.4.3 Computing Procedure 265
6.4.4 Case Study 267
BFG Consumption Flow of Hot Blast Stove 267
BFG Generation Flows 269
6.5 Multi-dimension Granular-Based Long-Term Prediction Intervals 272
6.5.1 Case Study 273
6.6 Discussion 276
References 279
Chapter 7: Parameter Estimation and Optimization 280
7.1 Introduction 280
7.2 Gradient-Based Methods 281
7.2.1 Gradient Descent 282
Batch Gradient Descent 284
Stochastic Gradient Descent 284
Mini-batch Gradient Descent 285
7.2.2 Newton Method 285
7.2.3 Quasi-Newton Method 288
Broyden-Fletcher-Goldfarb-Shanno (BFGS) Algorithm 289
L-BFGS Method 291
7.2.4 Conjugate Gradient Method 293
Nonlinear Conjugate Gradient Method 295
7.2.5 Illustration: A Gradient Grid Search Algorithm 297
7.3 Intelligent Optimization Algorithms 299
7.3.1 Genetic Algorithm 300
Selection Operator 301
Crossover Operator 302
Mutation Operator 302
7.3.2 Differential Evolution Algorithm 304
Initialization of the Population 305
Mutation with Differential Operators 305
Crossover 306
Selection 306
7.3.3 Particle Swarm Optimization Algorithm 308
7.3.4 Simulated Annealing Algorithm 311
7.4 Nonlinear Kalman-Filter Estimation 314
7.4.1 Extended Kalman-Filter 315
7.4.2 Unscented Kalman-Filter 317
7.4.3 Cubature Kalman-Filter 319
7.4.4 Nonlinear Kalman-Filters-Based Dual Estimation 321
7.4.5 Dual Estimation of Linear/Nonlinear Kalman-Filter 323
7.4.6 Case Study 325
7.5 Probabilistic Methods 328
7.5.1 Maximum Likelihood Method 328
MLE for Linear Regression Model 330
7.5.2 Bayesian Method 332
7.5.3 Variational Inference 334
7.5.4 Variational Relevance Vector Machine Based on Automatic Relevance Determination Kernel Functions 340
Preliminaries on Variational RVM (VRVM) 340
Model Specification for the VRVM-ARDK 342
Variational Inference for RVM-ARDK 343
Predictive Distribution and Model Training 347
7.5.5 Case Study 348
7.6 Parameter Optimization for LS-SVM Based on Noise Estimation 350
7.6.1 Hyper-parameters Optimization Based on the CG Method 351
7.6.2 Case Study 353
The Sinc Function 353
Industrial Application: Prediction Modeling for Industrial Gas Flow 354
7.7 Discussion 358
References 359
Chapter 8: Parallel Computing Considerations 362
8.1 Introduction 362
8.2 CUDA-Based Parallel Acceleration 364
8.2.1 What´s CUDA? 364
8.2.2 Computing Architecture of CUDA 365
8.2.3 CUDA Libraries 366
8.3 Hadoop-Based Distributed Computation 367
8.3.1 What´s Hadoop? 367
8.3.2 Computing Architecture of Hadoop 368
8.3.3 MapReduce 370
8.4 GPU Acceleration for Training EKF-Based Elman Networks 372
8.4.1 Case Study 374
BFG System Prediction 374
COG System Prediction 376
8.5 Online Parameter Optimization-Based Prediction by GPU Acceleration 378
8.5.1 Initialization and Sub-swarm Separation 379
Update of Velocity and Position 380
Parameter Adaptation and Process Communication 380
8.5.2 Case Study 381
Parallelization for Parameter Validation 381
Parallelization for Parameter Selection 383
Online Prediction for LDG System 384
8.6 Parallelized EKF Based on MapReduce Framework 387
8.6.1 Case Study 391
8.7 Discussion 393
References 394
Chapter 9: Data-Based Prediction for Energy Scheduling of Steel Industry 395
9.1 Introduction 395
9.2 A Prediction and Adjustment Method for By-product Gas Scheduling 396
9.2.1 Holder Level Prediction Models and Adjustment Method 398
Holder Level Prediction Model 399
Optimization of Holder Level Prediction Model 402
Optimal Adjustment Quantity Determination 405
9.2.2 Case Study 406
9.3 Interval Predictive Optimization for By-product Gas System in Steel Industry 413
9.3.1 Mixed Gaussian Kernel-Based PIs Construction 416
Mixed Gaussian Kernel-Based Method for Regression 416
Construction of Mixed Gaussian Kernel-Based PIs 417
9.3.2 A Novel Predictive Optimization Method 419
Prediction for Optimized Objectives 420
Rolling Optimization for Adjusting Solution 422
9.3.3 Case Study 424
9.4 A Two-Stage Method for Predicting and Scheduling Energy in an Oxygen/Nitrogen System of the Steel industry 429
9.4.1 GrC-Based Long-Term Prediction Model 431
Data Granulation 431
Long-Term Prediction Modeling 432
9.4.2 MILP-Based Scheduling Model for Oxygen/Nitrogen System 434
Objective 434
Constraints 434
9.4.3 Case study 437
Long-Term Prediction on Oxygen/Nitrogen Requirements 437
Energy Scheduling and Optimization 439
Oxygen Shortage/Nitrogen Shortage 440
Oxygen Surplus/Nitrogen Shortage 442
9.5 Discussion 442
References 444
Index 447