Deep Learning and Convolutional Neural Networks for Medical Imaging and Clinical Informatics (eBook)

Preface 6
Organization and Features 7
Contents 9
Part I Segmentation 12
1 Pancreas Segmentation in CT and MRI via Task-Specific Network Design and Recurrent Neural Contextual Learning 13
1.1 Introduction 14
1.2 Convolutional Neural Network for Pancreas Segmentation 15
1.2.1 Design of Network Architecture 15
1.2.2 Design of Model Training Strategy 16
1.2.3 Design of Loss Functions 18
1.2.4 Experimental Results 19
1.3 Recurrent Neural Network for Contextual Learning 22
1.4 Recurrent Neural Network 23
1.4.1 Bidirectional Contextual Regularization 25
1.4.2 Experimental Results 25
1.5 State-of-the-Art Methods for Pancreas Segmentation 27
1.6 Summary 29
References 30
2 Deep Learning for Muscle Pathology Image Analysis 32
2.1 Introduction 33
2.2 Muscle Perimysium Segmentation 33
2.2.1 Recurrent Neural Network 34
2.3 WSI Inflammatory Muscle Disease Subtype Classification 40
2.3.1 Methodology 40
2.4 Experimental Results 43
2.4.1 Dataset 43
2.4.2 Implementation Details 43
2.4.3 Evaluation of Different WSI Frameworks 44
2.4.4 Evaluation of Different Training Methods 46
2.4.5 Evaluation of Different Number of ROIs 46
2.4.6 Diagnosis Interpretation and Visualization 47
2.5 Summary 48
References 49
3 2D-Based Coarse-to-Fine Approaches for Small Target Segmentation in Abdominal CT Scans 51
3.1 Introduction 52
3.2 Related Work 53
3.3 A Step-Wise Coarse-to-Fine Approach for Medical Image Segmentation 54
3.3.1 Deep Segmentation Networks 55
3.3.2 Fixed-Point Optimization 55
3.3.3 Application to Pancreatic Cyst Segmentation 58
3.4 An End-to-End Coarse-to-Fine Approach for Medical Image Segmentation 60
3.4.1 Recurrent Saliency Transformation Network 60
3.4.2 Training and Testing 62
3.4.3 Application to Pancreatic Cyst Segmentation 63
3.5 Pancreas Segmentation Experiments 64
3.5.1 Dataset and Evaluation 64
3.5.2 Evaluation of the Step-Wise Coarse-to-Fine Approach 64
3.5.3 Evaluation of the End-to-End Coarse-to-Fine Approach 66
3.6 JHMI Multi-organ Segmentation Experiments 70
3.7 JHMI Pancreatic Cyst Segmentation Experiments 71
3.8 Conclusions 72
References 72
4 Volumetric Medical Image Segmentation: A 3D Deep Coarse-to-Fine Framework and Its Adversarial Examples 76
4.1 Introduction 77
4.2 Related Work 79
4.2.1 Deep Learning-Based Medical Image Segmentation 79
4.2.2 Adversarial Attacks and Defenses for Medical Image Segmentation Networks 81
4.3 Method 81
4.3.1 A 3D Coarse-to-Fine Framework for Medical Image Segmentation 81
4.3.2 3D Adversarial Examples 86
4.4 Experiments 87
4.4.1 Pancreas Segmentation 88
4.4.2 Adversarial Attack and Defense 93
4.5 Conclusion 96
References 96
5 Unsupervised Domain Adaptation of ConvNets for Medical Image Segmentation via Adversarial Learning 99
5.1 Introduction 100
5.2 Related Works 101
5.2.1 Feature-Level Adaptation 101
5.2.2 Pixel-Level Adaptation 102
5.3 Feature-Level Adaptation with Latent Space Alignment 103
5.3.1 Method 103
5.3.2 Experimental Results 107
5.4 Pixel-Level Adaptation with Image-to-Image Translation 112
5.4.1 Method 112
5.4.2 Experimental Results 115
5.5 Discussion 118
5.6 Conclusion 119
References 119
Part II Detection and Localization 122
6 Glaucoma Detection Based on Deep Learning Network in Fundus Image 123
6.1 Introduction 124
6.2 M-Net: Multi-label Segmentation Network 126
6.2.1 Multi-scale U-Shape Network 126
6.2.2 Side-Output Layer 128
6.2.3 Multi-label Loss Function 128
6.2.4 Polar Transformation 129
6.3 DENet: Disc-Aware Ensemble Network 130
6.3.1 Global Fundus Image Level 131
6.3.2 Optic Disc Region Level 132
6.4 Experiments 133
6.4.1 Implementation 133
6.4.2 Segmentation Evaluation 133
6.4.3 Glaucoma Screening Evaluation 136
6.4.4 REFUGE Challenge 138
6.5 Conclusion 139
References 140
7 Thoracic Disease Identification and Localization with Limited Supervision 142
7.1 Introduction 143
7.2 Related Work 145
7.3 Model 146
7.3.1 Image Model 146
7.3.2 Loss Function 147
7.3.3 Localization Generation 149
7.3.4 Training 150
7.4 Experiments 151
7.4.1 Disease Identification 151
7.4.2 Disease Localization 154
7.4.3 Qualitative Results 161
7.5 Conclusion 162
References 162
8 Deep Reinforcement Learning for Detecting Breast Lesions from DCE-MRI 165
8.1 Introduction 166
8.2 Literature Review 168
8.3 Methods 169
8.3.1 Data Set 169
8.3.2 Detection Method 170
8.3.3 Training 171
8.3.4 Inference 173
8.4 Experiments 173
8.4.1 Data Set 174
8.4.2 Experimental Setup 174
8.4.3 Experimental Results 175
8.5 Discussion 177
8.6 Conclusion and Future Work 177
References 178
9 Automatic Vertebra Labeling in Large-Scale Medical Images Using Deep Image-to-Image Network with Message Passing and Sparsity Regularization 181
9.1 Introduction 182
9.2 Methodology 185
9.2.1 The Deep Image-to-Image Network (DI2IN) for Spinal Centroid Localization 185
9.2.2 Probability Map Enhancement with Message Passing 187
9.2.3 Joint Refinement Using Shape-Based Dictionaries 189
9.3 Experiments 192
9.4 Conclusion 197
References 197
10 Anisotropic Hybrid Network for Cross-Dimension Transferable Feature Learning in 3D Medical Images 200
10.1 Introduction 201
10.2 Related Work 202
10.3 Anisotropic Hybrid Network 203
10.3.1 Learning a Multichannel 2D Feature Encoder 204
10.3.2 Transferring the Learned 2D Net to 3D AH-Net 205
10.3.3 Anisotropic Hybrid Decoder 208
10.4 Experimental Results 208
10.4.1 Breast Lesion Detection from DBT 209
10.4.2 Liver and Liver Tumor Segmentation from CT 213
10.5 Conclusion 216
References 216
Part III Various Applications 218
11 Deep Hashing and Its Application for Histopathology Image Analysis 219
11.1 Introduction 219
11.2 Deep Hashing 220
11.2.1 Pointwise-Based Hashing 220
11.2.2 Multiwise-Based Hashing 221
11.2.3 Pairwise-Based Hashing 222
11.3 Experimental Results and Discussion 229
11.3.1 Experimental Results 231
11.3.2 Discussion 233
11.4 Summary 234
References 235
12 Tumor Growth Prediction Using Convolutional Networks 238
12.1 Introduction 239
12.2 Group Learning Approach for Tumor Growth Prediction 240
12.2.1 Image Processing and Patch Extraction 242
12.2.2 Learning a Voxel-Wise Deep Representation 243
12.2.3 Learning a Predictive Model with Multi-source Features 244
12.2.4 Experiments and Results 245
12.3 Convolutional Invasion and Expansion Networks for Tumor Growth Prediction 247
12.3.1 Learning Invasion Network 248
12.3.2 Learning Expansion Network 249
12.3.3 Fusing Invasion and Expansion Networks 250
12.3.4 Personalizing Invasion and Expansion Networks 252
12.3.5 Predicting with Invasion and Expansion Networks 253
12.3.6 Experimental Methods and Results 253
12.4 Summary 257
References 257
13 Deep Spatial-Temporal Convolutional Neural Networks for Medical Image Restoration 260
13.1 Introduction 261
13.2 Related Work 262
13.2.1 Radiation Dose Reduction 263
13.2.2 Image Restoration 263
13.2.3 Spatial-Temporal Architecture 264
13.3 Methodology 265
13.3.1 Spatial-Temporal Patches 265
13.3.2 Deep Spatial-Temporal Network 265
13.4 Platform and Data Acquisition 267
13.4.1 Computational Platform 267
13.4.2 Datasets 267
13.4.3 Low Radiation Dose Simulation and Data Preprocessing 268
13.5 Experiments and Results 269
13.5.1 Evaluation Metrics 269
13.5.2 Spatial-Temporal Super-Resolution and Denoising 270
13.6 Conclusion 272
References 273
14 Generative Low-Dose CT Image Denoising 275
14.1 Introduction 276
14.2 Methods 279
14.2.1 Noise Reduction Model 279
14.2.2 WGAN 279
14.2.3 Perceptual Loss 280
14.2.4 Network Structures 280
14.2.5 Other Networks 282
14.3 Experiments 282
14.3.1 Experimental Datasets 282
14.3.2 Network Training 283
14.3.3 Network Convergence 283
14.3.4 Denoising Results 285
14.3.5 Quantitative Analysis 287
14.4 Discussions and Conclusion 291
References 293
15 Image Quality Assessment for Population Cardiac Magnetic Resonance Imaging 296
15.1 Introduction 297
15.2 Full LV Coverage Detection Method 300
15.2.1 Problem Formulation 300
15.2.2 Dataset Invariance 3D Intensity Representations 301
15.2.3 Fisher Discriminative 3D CNN Model 304
15.3 Materials and Metrics 306
15.3.1 CMR Acquisition Protocol and Annotation 306
15.3.2 Training and Testing Set Definitions 307
15.3.3 Learning Performance Metrics 309
15.4 Experiments and Results 309
15.4.1 Hyper Parameter Selection on UK Biobank 309
15.4.2 Dataset Adversarial Learning Performance 313
15.4.3 Intra-rater Agreement of Full LV Coverage Detection 315
15.4.4 Implementation Considerations 315
15.5 Conclusion 316
References 316
16 Agent-Based Methods for Medical Image Registration 319
16.1 Introduction 319
16.2 Background 320
16.2.1 Parametric Image Registration 321
16.2.2 Image Registration Using Deep Learning 321
16.2.3 Deep Reinforcement Learning 322
16.2.4 Special Euclidean Group SE(3) 322
16.3 Agent-Based Image Registration 322
16.3.1 Image Registration as an MDP 323
16.3.2 Action Space 324
16.3.3 Reward System 325
16.3.4 Agent Observation 325
16.3.5 Learning Policy with Supervised Learning 327
16.3.6 Multi-agent System 328
16.4 Agent-Based 3-D/3-D Image Registration 329
16.4.1 Implementation 329
16.4.2 Experiments and Results 331
16.5 Agent-Based 2-D/3-D Image Registration 333
16.5.1 Implementation 334
16.5.2 Experiments and Results 336
16.6 Discussion 339
References 340
17 Deep Learning for Functional Brain Connectivity: Are We There Yet? 342
17.1 Introduction 343
17.2 Related Work 344
17.3 Methods 346
17.3.1 fMRI Preprocessing and Feature Extraction 346
17.3.2 Ensemble Classification Approach 347
17.3.3 Deep Learning Models 348
17.4 Results 351
17.4.1 Independent Classifiers 351
17.4.2 Ensemble Classifiers 351
17.4.3 Deep Learning Classifiers 353
17.5 Discussion and Conclusions 355
References 358
Part IV Large-Scale Data Mining and Data Synthesis 361
18 ChestX-ray: Hospital-Scale Chest X-ray Database and Benchmarks on Weakly Supervised Classification and Localization of Common Thorax Diseases 362
18.1 Introduction 363
18.1.1 Recent Advances 365
18.2 Database Construction 366
18.2.1 Disease Label Mining 366
18.2.2 Evaluation on Mined Disease Labels 369
18.2.3 Chest X-ray Image Processing and Hand-Labeled Ground Truth 370
18.3 Applications on Constructed Database 371
18.3.1 Classification and Localization Framework 371
18.4 Evaluations 374
18.5 Extension to 14 Common Thorax Disease Labels 377
18.5.1 Evaluation of NLP Mined Labels 378
18.5.2 Benchmark Results 378
18.6 Summary 381
References 382
19 Automatic Classification and Reporting of Multiple Common Thorax Diseases Using Chest Radiographs 386
19.1 Introduction 387
19.2 Previous Works in CAD 389
19.3 Multi-level Attention in a Unified Framework 390
19.3.1 AETE: Attention on Text 390
19.3.2 SW-GAP: Attention on Image 392
19.3.3 Overall CNN-RNN Model 393
19.3.4 Joint Learning 394
19.4 Applications 395
19.4.1 Annotation of Chest X-Ray Images 395
19.4.2 Automatic Reporting of Thorax Diseases 395
19.5 Experiments 395
19.5.1 Datasets for Evaluation 395
19.5.2 Report Vocabulary 396
19.5.3 Evaluation Metrics 396
19.5.4 Details on Training 397
19.5.5 Evaluation on Image Annotation 397
19.5.6 Evaluation on Classification and Automated Reporting 400
19.6 Summary 403
References 403
20 Deep Lesion Graph in the Wild: Relationship Learning and Organization of Significant Radiology Image Findings in a Diverse Large-Scale Lesion Database 406
20.1 Introduction 407
20.2 Related Work 409
20.3 Dataset 410
20.4 Method 412
20.4.1 Supervision Cues 412
20.4.2 Learning Lesion Embeddings 414
20.4.3 Lesion Retrieval and Matching 416
20.5 Experiments 417
20.5.1 Implementation Details 418
20.5.2 Content-Based Lesion Retrieval 418
20.5.3 Intra-patient Lesion Matching 422
20.6 Conclusion and Future Work 423
References 425
21 Simultaneous Super-Resolution and Cross-Modality Synthesis in Magnetic Resonance Imaging 429
21.1 Introduction 430
21.2 Background 433
21.2.1 Image Degradation Model 433
21.2.2 Dictionary Learning 434
21.3 Method 434
21.3.1 Data Description 435
21.3.2 Gradient Feature Representation 436
21.3.3 Cross-Modality Dictionary Learning 436
21.3.4 Clustering-Based Globally Redundant Codes 437
21.4 Experiments 440
21.4.1 Dictionary Size 441
21.4.2 Sparsity 442
21.4.3 MRI Super-Resolution 443
21.4.4 Simultaneous Super-Resolution and Cross-Modality Synthesis 444
21.5 Conclusion 447
References 447
Appendix A Index 450
Index 450