Advanced Applied Deep Learning (eBook)

Table of Contents 5
About the Author 10
About the Technical Reviewer 11
Acknowledgments 12
Introduction 13
Chapter 1: Introduction and Development Environment Setup 15
GitHub Repository and Companion Website 17
Mathematical Level Required 17
Python Development Environment 18
Google Colab 19
Benefits and Drawbacks to Google Colab 22
Anaconda 23
Installing TensorFlow the Anaconda Way 25
Local Jupyter Notebooks 28
Benefits and Drawbacks to Anaconda 31
Docker Image 32
Benefits and Drawbacks to a Docker Image 38
Which Option Should You Choose? 39
Chapter 2: TensorFlow: Advanced Topics 41
Tensorflow Eager Execution 42
Enabling Eager Execution 43
Polynomial Fitting with Eager Execution 44
MNIST Classification with Eager Execution 48
TensorFlow and Numpy Compatibility 53
Hardware Acceleration 54
Checking the Availability of the GPU 54
Device Names 55
Explicit Device Placement 56
GPU Acceleration Demonstration: Matrix Multiplication 57
Effect of GPU Acceleration on the MNIST Example 59
Training Only Specific Layers 61
Training Only Specific Layers: An Example 62
Removing Layers 66
Keras Callback Functions 68
Custom Callback Class 69
Example of a Custom Callback Class 71
Save and Load Models 75
Save Your Weights Manually 81
Saving the Entire Model 82
Dataset Abstraction 82
Iterating Over a Dataset 85
Simple Batching 86
Simple Batching with the MNIST Dataset 87
Using tf.data.Dataset in Eager Execution Mode 90
Conclusions 91
Chapter 3: Fundamentals of Convolutional Neural Networks 92
Kernels and Filters 92
Convolution 94
Examples of Convolution 104
Pooling 112
Padding 117
Building Blocks of a CNN 118
Convolutional Layers 118
Pooling Layers 121
Stacking Layers Together 121
Number of Weights in a CNN 122
Convolutional Layer 122
Pooling Layer 123
Dense Layer 123
Example of a CNN: MNIST Dataset 123
Visualization of CNN Learning 128
Brief Digression: keras.backend.function() 128
Effect of Kernels 131
Effect of Max-Pooling 134
Chapter 4: Advanced CNNs and Transfer Learning 137
Convolution with Multiple Channels 137
History and Basics of Inception Networks 141
Inception Module: Naïve Version 143
Number of Parameters in the Naïve Inception Module 144
Inception Module with Dimension Reduction 145
Multiple Cost Functions: GoogLeNet 146
Example of Inception Modules in Keras 148
Digression: Custom Losses in Keras 151
How To Use Pre-Trained Networks 153
Transfer Learning: An Introduction 157
A Dog and Cat Problem 161
Classical Approach to Transfer Learning 162
Experimentation with Transfer Learning 169
Chapter 5: Cost Functions and Style Transfer 173
Components of a Neural Network Model 173
Training Seen as an Optimization Problem 174
A Concrete Example: Linear Regression 176
The Cost Function 177
Mathematical Notation 177
Typical Cost Functions 178
Mean Square Error 178
Intuitive Explanation 179
MSE as the Second Moment of a Moment-Generating Function 179
Cross-Entropy 181
Self-Information or Suprisal of an Event 181
Suprisal Associated with an Event X 183
Cross-Entropy 183
Cross-Entropy for Binary Classification 185
Cost Functions: A Final Word 188
Neural Style Transfer 188
The Mathematics Behind NST 190
An Example of Style Transfer in Keras 195
NST with Silhouettes 202
Masking 204
Chapter 6: Object Classification: An Introduction 206
What Is Object Localization? 207
Most Important Available Datasets 210
Intersect Over Union (IoU) 211
A Naïve Approach to Solving Object Localization (Sliding Window Approach) 213
Problems and Limitations the with Sliding Window Approach 215
Classification and Localization 222
Region-Based CNN (R-CNN) 224
Fast R-CNN 228
Faster R-CNN 230
Chapter 7: Object Localization: An Implementation in Python 232
The You Only Look Once (YOLO) Method 233
How YOLO Works 234
Dividing the Image Into Cells 234
YOLOv2 (Also Known As YOLO9000) 237
YOLOv3 238
Non-Maxima Suppression 239
Loss Function 239
Classification Loss 240
Localization Loss 240
Confidence Loss 240
Total Loss Function 241
YOLO Implementation in Python and OpenCV 242
Darknet Implementation of YOLO 242
Testing Object Detection with Darknet 244
Training a Model for YOLO for Your Specific Images 251
Concluding Remarks 252
Chapter 8: Histology Tissue Classification 253
Data Analysis and Preparation 254
Model Building 263
Data Augmentation 274
Horizontal and Vertical Shifts 276
Flipping Images Vertically 279
Randomly Rotating Images 279
Zooming in Images 282
Putting All Together 282
VGG16 with Data Augmentation 283
The fit() Function 283
The fit_generator() Function 284
The train_on_batch() Function 285
Training the Network 286
And Now Have Fun… 287
Index 288