Machine Learning with the Raspberry Pi (eBook)

Table of Contents 4
About the Author 8
About the Technical Reviewer 9
Chapter 1: Introduction to machine learning (ML) with the Raspberry Pi (RasPi) 10
RasPi introduction 10
Writing the Raspbian Image to a micro SD card 13
Mandatory configurations 15
Optional configurations 21
Updating and upgrading the Raspbian distribution 24
Python virtual environment 25
Installing a Python virtual environment 26
Installing dependencies 31
ML facts 33
ML basics 34
Linear prediction and classification 35
Iris demonstration – Part 1 38
Iris demonstration – Part 2 44
Iris demonstration – Part 3 49
Chapter 2: Exploration of ML data models: Part 1 57
Installing OpenCV 4 57
Download OpenCV 4 source code 59
Building the OpenCV software 60
Seaborn data visualization library 67
Scatter plot 70
Facet grid plot 73
Box plot 75
Strip plot 76
Violin plot 78
KDE plot 80
Pair plots 82
Underlying big principle 85
Linear regression 85
LR demonstration 86
Logistic regression 89
LogR model development 93
LogR demonstration 94
Naive Bayes 99
Brief review of the Bayes’ theorem 100
Preparing data for use by the Naive Bayes model 101
Naive Bayes model example 102
Pros and cons 105
Gaussian Naive Bayes 107
Gaussian Naive Bayes (GNB) demonstration 108
k-nearest neighbor (k-NN) model 109
KNN demonstration 110
Decision tree classifier 114
Decision tree algorithm 115
Information gain 117
Split criterion 117
Measuring information 118
Properties of entropy 119
Information gain example 120
Gini index 125
Simple Gini index example 126
Gain ratio 129
Intrinsic information 129
Definition of gain ratio 130
Decision tree classifier demonstration with scikit-learn 130
Visualizing the decision tree 135
Optimizing a decision tree 139
Pros and cons for decision trees 140
Pros 140
Cons 141
Chapter 3: Exploration of ML data models: Part 2 142
Principal component analysis 143
PCA script discussion 144
PCA demonstration 153
When to use PCA 157
Linear discriminant analysis 157
LDA script discussion 159
LDA demonstration 165
Comparison of PCA and LDA 168
Support vector machines 169
SVM demonstration – Part 1 173
SVM demonstration – Part 2 176
Learning vector quantization 184
LVQ basic concepts 185
Euclidean distance 185
Best matching unit 186
Training codebook vectors 186
LVQ demonstration 187
Bagging and random forests 197
Introduction to bagging and random forest 197
Bootstrap aggregation (bagging) 198
Random forest 199
Performance estimation and variable importance 200
Bootstrap resampling demonstration 200
Bagging demonstration 202
Random forest demonstration 211
Chapter 4: Preparation for deep learning 220
DL basics 220
Machine learning from data patterns 221
Linear classifier 223
Loss functions 228
Different types of loss functions 228
Optimizer algorithm 232
Deep dive into the gradient descent algorithm 235
Artificial neural network 242
How ANNs are trained and function 245
Practical ANN example 249
Complex ANN example 252
Modifying weight values 256
Practical ANN weight modification example 265
Some issues with ANN learning 266
ANN Python demonstration – Part 1 269
ANN Python demonstration – Part 2 274
Chapter 5: Practical deep learning ANN demonstrations 285
Parts list 286
Recognizing handwritten number demonstration 286
Project history and preparatory details 290
Adjusting the input datasets 299
Interpreting ANN output data values 301
Creating an ANN that does handwritten number recognition 303
Initial ANN training script demonstration 305
ANN test script demonstration 307
ANN test script demonstration using the full training dataset 315
Recognizing your own handwritten numbers 319
Installing the Pi Camera 320
Installing the Pi Camera software 324
Handwritten number recognition demonstration 325
Handwritten number recognition using Keras 330
Introduction to Keras 330
Installing Keras 331
Downloading the dataset and creating a model 332
Chapter 6: CNN demonstrations 341
Parts list 341
Introduction to the CNN model 342
History and evolution of the CNN 348
Fashion MNIST demonstration 364
More complex Fashion MNIST demonstration 375
VGG Fashion MNIST demonstration 379
Jason’s Fashion MNIST demonstration 384
Chapter 7: Predictions using ANNs and CNNs 392
Pima Indian Diabetes demonstration 393
Background for the Pima Indian Diabetes study 393
Preparing the data 394
Using the scikit-learn library with Keras 413
Grid search with Keras and scikit-learn 416
Housing price regression predictor demonstration 421
Preprocessing the data 422
The baseline model 426
Improved baseline model 430
Another improved baseline model 433
Predictions using CNNs 436
Univariate time series CNN model 438
Preprocessing the dataset 438
Create a CNN model 441
Multivariate time series CNN model 445
Multiple input series 446
Preprocessing the dataset 447
Chapter 8: Predictions using CNNs and MLPs for medical research 457
Parts list 458
Downloading the breast cancer histology Image dataset 459
Preparing the project environment 463
Configuration script 464
Building the dataset 465
Running the build dataset script 468
The CNN model 470
Training and testing script 474
Running the training and testing script 482
Evaluating the results with a discussion of sensitivity, specificity, and AUROC curves 486
What is sensitivity? 487
What is specificity? 488
What are the differences between sensitivity and specificity and how are they used? 488
Using a MLP model for breast cancer prediction 495
Running the MLP script 500
Chapter 9: Reinforcement learning 505
Markov decision process 507
Discounted future reward 509
Q-learning 510
Q-learning example 513
Manual Q-learning experiments 521
Q-learning demonstration with a Python script 527
Running the script 533
Q-learning in a hostile environment demonstration 535
Running the script and evaluating the results 540
Q-learning in a hostile environment with a priori knowledge demonstration 543
Running the script and evaluating the results 548
Q-learning and neural networks 551
Index 558