MATLAB Machine Learning (eBook)

Michael Paluszek is the co-author of MATLAB Recipes published by Apress. He is President of Princeton Satellite Systems, Inc. (PSS) in Plainsboro, New Jersey. Mr. Paluszek founded PSS in 1992 to provide aerospace consulting services. He used MATLAB to develop the control system and simulation for the Indostar-1 geosynschronous communications satellite, resulting in the launch of PSS' first commercial MATLAB toolbox, the Spacecraft Control Toolbox, in 1995. Since then he has developed toolboxes and software packages for aircraft, submarines, robotics, and fusion propulsion, resulting in PSS' current extensive product line. He is currently leading an Army research contract for precision attitude control of small satellites and working with the Princeton Plasma Physics Laboratory on a compact nuclear fusion reactor for energy generation and propulsion. Prior to founding PSS, Mr. Paluszek was an engineer at GE Astro Space in East Windsor, NJ. At GE he designed the Global Geospace Science Polar despun platform control system and led the design of the GPS IIR attitude control system, the Inmarsat-3 attitude control systems and the Mars Observer delta-V control system, leveraging MATLAB for control design. Mr. Paluszek also worked on the attitude determination system for the DMSP meteorological satellites. Mr. Paluszek flew communication satellites on over twelve satellite launches, including the GSTAR III recovery, the first transfer of a satellite to an operational orbit using electric thrusters. At Draper Laboratory Mr. Paluszek worked on the Space Shuttle, Space Station and submarine navigation. His Space Station work included designing of Control Moment Gyro based control systems for attitude control. Mr. Paluszek received his bachelors in Electrical Engineering, and master's and engineer’s degrees in Aeronautics and Astronautics from the Massachusetts Institute of Technology. He is author of numerous papers and has over a dozen U.S. Patents.Stephanie Thomas is the co-author of MATLAB Recipes, published by Apress. She received her bachelor's and master's degrees in Aeronautics and Astronautics from the Massachusetts Institute of Technology in 1999 and 2001. Ms. Thomas was introduced to PSS' Spacecraft Control Toolbox for MATLAB during a summer internship in 1996 and has been using MATLAB for aerospace analysis ever since. She built a simulation of a lunar transfer vehicle in C++, LunarPilot, during the same internship. In her nearly 20 years of MATLAB experience, she has developed many software tools including the Solar Sail Module for the Spacecraft Control Toolbox; a proximity satellite operations toolbox for the Air Force; collision monitoring Simulink blocks for the Prisma satellite mission; and launch vehicle analysis tools in MATLAB and Java, to name a few. She has developed novel methods for space situation assessment such as a numeric approach to assessing the general rendezvous problem between any two satellites implemented in both MATLAB and C++. Ms. Thomas has contributed to PSS' Attitude and Orbit Control textbook, featuring examples using the Spacecraft Control Toolbox, and written many software User's Guides. She has conducted SCT training for engineers from diverse locales such as Australia, Canada, Brazil, and Thailand and has performed MATLAB consulting for NASA, the Air Force, and the European Space Agency.

Contents at a Glance 5
Contents 6
About the Authors 14
About the Technical Reviewer 16
Introduction 17
Part I Introduction to Machine Learning 18
Chapter 1:An Overview of Machine Learning 19
1.1 Introduction 19
1.2 Elements of Machine Learning 20
1.2.1 Data 20
1.2.2 Models 20
1.2.3 Training 21
1.2.3.1 Supervised Learning 21
1.2.3.2 Unsupervised Learning 21
1.2.3.3 Semisupervised Learning 21
1.2.3.4 Online Learning 21
1.3 The Learning Machine 22
1.4 Taxonomy of Machine Learning 23
1.5 Autonomous Learning Methods 24
1.5.1 Regression 24
1.5.2 Neural Nets 27
1.5.3 Support Vector Machines 28
1.5.4 Decision Trees 28
1.5.5 Expert System 29
References 31
Chapter 2:The History of Autonomous Learning 32
2.1 Introduction 32
2.2 Artificial Intelligence 32
2.3 Learning Control 34
2.4 Machine Learning 36
2.5 The Future 37
References 38
Chapter 3:Software for Machine Learning 39
3.1 Autonomous Learning Software 39
3.2 Commercial MATLAB Software 39
3.2.1 MathWorks Products 39
3.2.1.1 Statistics and Machine Learning Toolbox 40
3.2.1.2 Neural Network Toolbox 40
3.2.1.3 Computer Vision System Toolbox 40
3.2.1.4 System Identification Toolbox 41
3.2.2 Princeton Satellite Systems Products 41
3.2.2.1 Core Control Toolbox 41
3.2.2.2 Target Tracking 41
3.3 MATLAB Open-Source Resources 42
3.3.1 Deep Learn Toolbox 42
3.3.2 Deep Neural Network 42
3.3.3 MatConvNet 42
3.4 Products for Machine Learning 42
3.4.1 R 42
3.4.2 scikit-learn 42
3.4.3 LIBSVM 43
3.5 Products for Optimization 43
3.5.1 LOQO 43
3.5.2 SNOPT 43
3.5.3 GLPK 44
3.5.4 CVX 44
3.5.5 SeDuMi 44
3.5.6 YALMIP 44
References 45
Part II MATLAB Recipes for Machine Learning 46
Chapter 4:Representation of Data for Machine Learning in MATLAB 47
4.1 Introduction to MATLAB Data Types 47
4.1.1 Matrices 47
4.1.2 Cell Arrays 48
4.1.3 Data Structures 49
4.1.4 Numerics 50
4.1.5 Images 50
4.1.6 Datastore 52
4.1.7 Tall Arrays 53
4.1.8 Sparse Matrices 54
4.1.9 Tables and Categoricals 54
4.1.10 Large MAT-Files 55
4.2 Initializing a Data Structure Using Parameters 56
4.2.1 Problem 56
4.2.2 Solution 56
4.2.3 How It Works 56
4.3 Performing mapreduce on an Image Datastore 58
4.3.1 Problem 58
4.3.2 Solution 58
4.3.3 How It Works 58
4.4 Creating a Table from a File 60
Summary 60
Chapter 5MATLAB Graphics: 61
5.1 Two-Dimensional Line Plots 61
5.1.1 Problem 61
5.1.2 Solution 61
5.1.3 How It Works 62
5.2 General 2D Graphics 66
5.2.1 Problem 66
5.2.2 Solution 66
5.2.3 How It Works 66
5.3 Custom 2D Diagrams 70
5.3.1 Problem 70
5.3.2 Solution 70
5.3.3 How It Works 71
5.4 Three-Dimensional Box 77
5.4.1 Problem 77
5.4.2 Solution 77
5.4.3 How It Works 77
5.5 Draw a 3D Object with a Texture 79
5.5.1 Problem 79
5.5.2 Solution 80
5.5.3 How It Works 80
5.6 General 3D Graphics 82
5.6.1 Problem 82
5.6.2 Solution 82
5.6.3 How It Works 83
5.7 Building a Graphical User Interface 84
5.7.1 Problem 84
5.7.2 Solution 84
5.7.3 How It Works 84
Summary 96
Chapter 6:Machine Learning Examples in MATLAB 97
6.1 Introduction 97
6.2 Machine Learning 97
6.2.1 Neural Networks 97
6.2.2 Face Recognition 98
6.2.3 Data Classification 98
6.3 Control 98
6.3.1 Kalman Filters 98
6.3.2 Adaptive Control 99
6.4 Artificial Intelligence 99
6.4.1 Autonomous Driving and Target Tracking 100
Chapter 7:Face Recognition with Deep Learning 101
7.1 Obtain Data Online: For Training a Neural Network 104
7.1.1 Problem 104
7.1.2 Solution 105
7.1.3 How It Works 105
7.2 Generating Data for Training a Neural Net 105
7.2.1 Problem 105
7.2.2 Solution 105
7.2.3 How It Works 105
7.3 Convolution 109
7.3.1 Problem 109
7.3.2 Solution 110
7.3.3 How It Works 110
7.4 Convolution Layer 112
7.4.1 Problem 112
7.4.2 Solution 112
7.4.3 How It Works 112
7.5 Pooling 115
7.5.1 Problem 115
7.5.2 Solution 115
7.5.3 How It Works 115
7.6 Fully Connected Layer 116
7.6.1 Problem 116
7.6.2 Solution 116
7.6.3 How It Works 116
7.7 Determining the Probability 118
7.7.1 Problem 118
7.7.2 Solution 118
7.7.3 How It Works 119
7.8 Test the Neural Network 120
7.8.1 Problem 120
7.8.2 Solution 120
7.8.3 How It Works 120
7.9 Recognizing an Image 121
7.9.1 Problem 121
7.9.2 Solution 121
7.9.3 How It Works 122
Summary 123
Reference 124
Chapter 8:Data Classification 125
8.1 Generate Classification Test Data 125
8.1.1 Problem 125
8.1.2 Solution 125
8.1.3 How It Works 125
8.2 Drawing Decision Trees 128
8.2.1 Problem 128
8.2.2 Solution 128
8.2.3 How It Works 128
8.3 Decision Tree Implementation 132
8.3.1 Problem 132
8.3.2 Solution 132
8.3.3 How It Works 132
8.4 Implementing a Decision Tree 136
8.4.1 Problem 136
8.4.2 Solution 136
8.4.3 How It Works 136
8.5 Creating a Hand-Made Decision Tree 141
8.5.1 Problem 141
8.5.2 Solution 141
8.5.3 How It Works 141
8.6 Training and Testing the Decision Tree 146
8.6.1 Problem 146
8.6.2 Solution 146
8.6.3 How It Works 146
Summary 152
Reference 153
Chapter 9:Classification of Numbers Using Neural Networks 154
9.1 Generate Test Images with Defects 154
9.1.1 Problem 154
9.1.2 Solution 154
9.1.3 How It Works 155
9.2 Create the Neural Net Tool 157
9.2.1 Problem 157
9.2.2 Solution 158
9.2.3 How It Works 158
9.3 Train a Network with One Output Node 167
9.3.1 Problem 167
9.3.2 Solution 168
9.3.3 How It Works 169
9.4 Testing the Neural Network 172
9.4.1 Problem 172
9.4.2 Solution 172
9.4.3 How It Works 172
9.5 Train a Network with Multiple Output Nodes 173
9.5.1 Problem 173
9.5.2 Solution 173
9.5.3 How It Works 173
Summary 177
References 178
Chapter 10:Kalman Filters 179
10.1 A State Estimator 180
10.1.1 Problem 180
10.1.2 Solution 185
10.1.3 How It Works 186
10.1.4 Conventional Kalman Filter 190
10.2 Using the Unscented Kalman Filter for StateEstimation 200
10.2.1 Problem 200
10.2.2 Solution 200
10.2.3 How It Works 200
10.3 Using the UKF for Parameter Estimation 207
10.3.1 Problem 207
10.3.2 Solution 207
10.3.3 How It Works 207
Summary 213
References 215
Chapter 11:Adaptive Control 216
11.1 Self-Tuning: Finding the Frequency of an Oscillator 217
11.1.1 Problem 219
11.1.2 Solution 219
11.1.3 How It Works 219
11.2 Model Reference Adaptive Control 226
11.2.1 Generating a Square Wave Input 226
11.2.1.1 Problem 226
11.2.1.2 Solution 226
11.2.1.3 How It Works 226
11.2.2 Implement Model Reference Adaptive Control 228
11.2.2.1 Problem 228
11.2.2.2 Solution 228
11.2.2.3 How It Works 228
11.2.3 Demonstrate MRAC for a Rotor 231
11.2.3.1 Problem 231
11.2.3.2 Solution 231
11.2.3.3 How It Works 231
11.3 Longitudinal Control of an Aircraft 234
11.3.1 Write the Differential Equations for the LongitudinalMotion of an Aircraft 234
11.3.1.1 Problem 234
11.3.1.2 Solution 234
11.3.1.3 How It Works 234
11.3.2 Numerically Finding Equilibrium 240
11.3.2.1 Problem 240
11.3.2.2 Solution 240
11.3.2.3 How It Works 240
11.3.3 Numerical Simulation of the Aircraft 242
11.3.3.1 Problem 242
11.3.3.2 Solution 242
11.3.3.3 How It Works 242
11.3.4 Find a Limiting and Scaling function for a Neural Net 244
11.3.4.1 Problem 244
11.3.4.2 Solution 244
11.3.4.3 How It Works 244
11.3.5 Find a Neural Net for the Learning Control 245
11.3.5.1 Problem 245
11.3.5.2 Solution 245
11.3.5.3 How It Works 245
11.3.6 Enumerate All Sets of Inputs 249
11.3.6.1 Problem 249
11.3.6.2 Solution 249
11.3.6.3 How It Works 250
11.3.7 Write a General Neural Net Function 251
11.3.7.1 Problem 251
11.3.7.2 Solution 251
11.3.7.3 How It Works 251
11.3.8 Implement PID Control 256
11.3.8.1 Problem 256
11.3.8.2 Solution 256
11.3.8.3 How It Works 256
11.3.9 Demonstrate PID control of Pitch for the Aircraft 260
11.3.9.1 Problem 260
11.3.9.2 Solution 260
11.3.9.3 How It Works 260
11.3.10 Create the Neural Net for the Pitch Dynamics 265
11.3.10.1 Problem 265
11.3.10.2 Solution 265
11.3.10.3 How It Works 265
11.3.11 Demonstrate the Controller in a Nonlinear Simulation 268
11.3.11.1 Problem 268
11.3.11.2 Solution 268
11.3.11.3 How It Works 268
11.4 Ship Steering: Implement Gain Scheduling for Steering Control of a Ship 270
11.4.1 Problem 270
11.4.2 Solution 270
11.4.3 How It Works 271
Summary 276
References 277
Chapter12:Autonomous Driving 278
12.1 Modeling the Automobile Radar 278
12.1.1 Problem 278
12.1.2 How It Works 278
12.1.3 Solution 279
12.2 Automobile Autonomous Passing Control 283
12.2.1 Problem 283
12.2.2 Solution 283
12.2.3 How It Works 283
12.3 Automobile Dynamics 285
12.3.1 Problem 285
12.3.2 How It Works 285
12.3.3 Solution 288
12.4 Automobile Simulation and the Kalman Filter 290
12.4.1 Problem 290
12.4.2 Solution 290
12.4.3 How It Works 290
12.5 Perform MHT on the Radar Data 297
12.5.1 Problem 297
12.5.2 Solution 297
12.5.3 How It Works 301
12.5.4 Hypothesis Formation 310
12.5.4.1 Problem 310
12.5.4.2 Solution 310
12.5.4.3 How It Works 310
12.5.5 Track Pruning 317
12.5.5.1 Problem 317
12.5.5.2 Solution 317
12.5.5.3 How It Works 317
12.5.5.4 Simulation 321
Summary 329
References 331
Index 332