Deep Learning in Mining of Visual Content (eBook)

Preface 7
Contents 9
Acronyms 12
List of Figures 13
1 Introduction 16
2 Supervised Learning Problem Formulation 19
2.1 Supervised Learning 19
2.2 Classification and Regression 20
2.3 Evaluation Metrics 21
2.3.1 Confusion Matrix 21
2.3.2 Metrics 23
2.3.3 AUC-ROC Curve 24
2.4 Conclusion 25
3 Neural Networks from Scratch 26
3.1 Formal Neuron 26
3.2 Artificial Neural Networks and Deep Neural Networks 29
3.3 Learning a Neural Network 30
3.3.1 Loss Function 31
3.3.2 One-Hot Encoding 31
3.3.3 Softmax or How to Transform Outputs into Probabilities 32
3.3.4 Cross-Entropy 32
3.4 Conclusion 33
4 Optimization Methods 34
4.1 Gradient Descent 34
4.2 Stochastic Gradient Descent 36
4.3 Momentum Based SGD 37
4.4 Nesterov Accelerated Gradient Descent 38
4.5 Adaptative Learning Rate 38
4.6 Extensions of Gradient Descent 40
4.7 Gradient Estimation in Neural Networks 41
4.7.1 A Simple Example 41
4.7.2 General Case: Backpropagation Algorithm 43
4.8 Conclusion 45
5 Deep in the Wild 47
5.1 Introduction 47
5.2 Convolution 48
5.3 Sub-sampling 54
5.3.1 Image Sampling 54
5.3.2 Sub-sampling of Images and Features 57
5.4 Conclusion 58
6 Convolutional Neural Networks as Image Analysis Tool 61
6.1 General Principles 61
6.2 Convolutional Layers 62
6.3 Max-Pooling Layers 64
6.4 Dropout 65
6.5 Some Well-Known CNNs Architectures 66
6.5.1 LeNet Architecture and MNIST Dataset 66
6.5.2 AlexNet Architecture 67
6.5.3 GoogLeNet 68
6.5.4 Other Important Architectures 68
6.6 Conclusion 70
7 Dynamic Content Mining 71
7.1 Hidden Markov Models 71
7.1.1 Likelihood Computation 73
7.1.2 Decoding: The Viterbi Algorithm 74
7.1.3 Learning an HMM 75
7.2 Recurrent Neural Networks 77
7.2.1 Definition 77
7.2.2 Training an RNN 79
7.3 Long-Short Term Memory Networks 80
7.3.1 The Model 80
7.4 Conclusion 81
8 Case Study for Digital Cultural Content Mining 82
8.1 Introduction 82
8.2 Visual Saliency 84
8.2.1 Top-Down Saliency Maps Built Upon Gaze Fixations 84
8.2.2 Co-saliency Propagation 86
8.3 Saliency in Deep CNNs 88
8.3.1 Attentional Mechanisms in Deep NNs 88
8.3.2 Squeeze-and-Excitation Networks 88
8.3.3 Double Attention Networks (A2-Nets) 88
8.3.4 Visual Saliency Propagation 90
8.3.4.1 Saliency in Pooling Layers 90
8.3.4.2 Saliency in the Dropout Layers 93
8.3.4.3 Use of Saliency in Backward Propagation 94
8.4 Conclusion 96
9 Introducing Domain Knowledge 97
9.1 Introduction 97
9.2 Domain Knowledge 98
9.2.1 Imaging Modalities 98
9.2.2 Selection of Brain ROI 100
9.2.3 Alignment of Different Imaging Modalities 101
9.3 Siamese Deep NNs for Fusion of Modalities 103
9.3.1 ``2D+?'' Approach for Classification of Brain Scans for AD Detection 104
9.3.2 Siamese Architecture 105
9.4 Conclusion 106
Conclusion 108
Glossary 109
References 110