Artificial Neural Networks with Java (eBook)

Table of Contents 5
About the Author 11
About the Technical Reviewer 12
Acknowledgments 13
Introduction 14
Chapter 1: Learning About Neural Networks 17
Biological and Artificial Neurons 18
Activation Functions 19
Summary 21
Chapter 2: Internal Mechanics of Neural Network Processing 22
Function to Be Approximated 22
Network Architecture 24
Forward-Pass Calculation 25
Input Record 1 26
Input Record 2 27
Input Record 3 28
Input Record 4 29
Backpropagation-Pass Calculations 30
Function Derivative and Function Divergent 31
Most Commonly Used Function Derivatives 32
Summary 34
Chapter 3: Manual Neural Network Processing 35
Example 1: Manual Approximation of a Function at a Single Point 35
Building the Neural Network 36
Forward-Pass Calculation 38
Hidden Layers 39
Output Layer 39
Backward-Pass Calculation 41
Calculating Weight Adjustments for the Output Layer Neurons 41
Calculating Adjustment for 41
Calculating the Adjustment for 42
Calculating the Adjustment for 44
Calculating Weight Adjustments for Hidden-Layer Neurons 45
Calculating the Adjustment for 45
Calculating the Adjustment for 46
Calculating the Adjustment for 47
Calculating the Adjustment for 48
Calculating the Adjustment for 49
Calculating the Adjustment for 50
Updating Network Biases 50
Going Back to the Forward Pass 52
Hidden Layers 52
Output Layer 53
Matrix Form of Network Calculation 56
Digging Deeper 56
Mini-Batches and Stochastic Gradient 59
Summary 60
Chapter 4: Configuring Your Development Environment 61
Installing the Java 11 Environment on Your Windows Machine 61
Installing the NetBeans IDE 64
Installing the Encog Java Framework 65
Installing the XChart Package 66
Summary 67
Chapter 5: Neural Network Development Using the Java Encog Framework 68
Example 2: Function Approximation Using the  Java Environment 68
Network Architecture 70
Normalizing the Input Data Sets 71
Building the Java Program That Normalizes Both Data Sets 71
Building the Neural Network Processing Program 82
Program Code 90
Debugging and Executing the Program 113
Processing Results for the Training Method 114
Testing the Network 115
Testing Results 119
Digging Deeper 120
Summary 121
Chapter 6: Neural Network Prediction Outside the Training Range 122
Example 3a: Approximating Periodic Functions Outside of the Training Range 123
Network Architecture for Example 3a 127
Program Code for Example 3a 127
Testing the Network 145
Example 3b: Correct Way of Approximating Periodic Functions Outside the Training Range 147
Preparing the Training Data 147
Network Architecture for Example 3b 150
Program Code for Example 3b 151
Training Results for Example 3b 173
Testing Results for Example 3b 175
Summary 176
Chapter 7: Processing Complex Periodic Functions 177
Example 4: Approximation of a Complex Periodic Function 177
Data Preparation 180
Reflecting Function Topology in the Data 181
Network Architecture 188
Program Code 188
Training the Network 212
Testing the Network 214
Digging Deeper 217
Summary 218
Chapter 8: Approximating Noncontinuous Functions 219
Example 5: Approximating Noncontinuous Functions 219
Network Architecture 223
Program Code 224
Code Fragments for the Training Process 238
Unsatisfactory Training Results 242
Approximating the Noncontinuous Function Using the Micro-Bach Method 244
Program Code for Micro-Batch Processing 245
Program Code for the getChart() Method 269
Code Fragment 1 of the Training Method 274
Code Fragment 2 of the Training Method 275
Training Results for the Micro-Batch Method 281
Test Processing Logic 287
Testing Results for the Micro-Batch Method 291
Digging Deeper 293
Summary 300
Chapter 9: Approximating Continuous Functions with Complex Topology 301
Example 5a: Approximation of a Continuous Function with Complex Topology Using the Conventional Network Process 301
Network Architecture for Example 5a 304
Program Code for Example 5a 305
Training Processing Results for Example 5a 319
Approximation of a Continuous Function with Complex Topology Using the Micro-Batch Method 322
Testing Processing for Example 5a 326
Example 5b: Approximation of Spiral-Like Functions 352
Network Architecture for Example 5b 356
Program Code for Example 5b 357
Approximation of the Same Function Using the Micro-Batch Method 374
Summary 404
Chapter 10: Using Neural Networks to Classify Objects 405
Example 6: Classification of Records 405
Training Data Set 407
Network Architecture 411
Testing Data Set 411
Program Code for Data Normalization 413
Program Code for Classification 419
Training Results 448
Testing Results 458
Summary 459
Chapter 11: The Importance of Selecting the Correct Model 460
Example 7: Predicting Next Month’s Stock Market Price 460
Including Function Topology in the Data Set 468
Building Micro-Batch Files 470
Network Architecture 476
Program Code 477
Training Process 511
Training Results 513
Testing Data Set 520
Testing Logic 525
Testing Results 535
Analyzing the Testing Results 538
Summary 540
Chapter 12: Approximation of Functions in 3D Space 541
Example 8: Approximation of Functions in 3D Space 542
Data Preparation 542
Network Architecture 547
Program Code 548
Processing Results 563
Summary 571
Index 572