Applied Deep Learning (eBook)

Umberto is currently the head of Innovation in BI & Analytics at a leading health insurance company in Switzerland, where he leads several strategic initiatives that deal with AI, new technologies and machine learning. He worked as data scientist and lead modeller in several big projects in healthcare and has extensive hands-on experience in programming and designing algorithms. Before that he managed projects in BI and DWH enabling data driven solutions to be implemented in complicated productive environments. He worked extensively with neural networks the last two years and applied deep learning to several problems linked to insurance and client behaviour (like customer churning). He presented his results on deep learning at international conferences and internally gained a reputation for his huge experience with Python and deep learning. 

Chapter 1:  Introduction Chapter Goal: Describe the book, the TensorFlow infrastructure, give instructions on how to setup a system for deep learning projectsNo of pages    :  30-50Sub -Topics1. Goal of the book2. Prerequisites3. TensorFlow Jupyter Notebooks introduction4. How to setup a computer to follow the book (docker image?)5. Tips for TensorFlow development and libraries needed (numpy, matplotlib, etc.)6. The problem of vectorization of code and calculations7. Additional resourcesChapter 2:  Single NeuronsChapter Goal: Describe what you can achieve with neural networks with just one neuron.No of pages: 50-70Sub -Topics8. Overview of different parts of a neuron9. Activation functions (ReLu, sigmoid, modified ReLu, etc.) and their difference (which one is for which task better)10. The new google activation function SWISH (https://arxiv.org/abs/1710.05941?utm_campaign=Artificial%2BIntelligence%2Band%2BDeep%2BLearning%2BWeekly&utm_medium=email&utm_source=Artificial_Intelligence_and_Deep_Learning_Weekly_35) 11. Optimization algorithm discussion (gradient descent)12. Linear regression13. Basic Tensorflow introduction14. Logistic regression15. Regression (linear and logistic) with tensorflow16. Practical case discussed in details17. The difference between regression and classification for one neuron18. Tips for TensorFlow implementationChapter 3:  Fully connected Neural Network with more neuronsChapter Goal: Describe what is a fully connected neural network and how to implement one (with one or more layers, etc.), and how to perform classification (binary and multi-class and regression)No of pages: 30-50Sub -Topics1. What is a tensor2. Dimensions of involved tensors (weights, input, etc.) (with tips on TensorFlow implementation)3. Distinctions between features and labels4. Problem of initialization of weights (random, constant, zeros, etc.)5. Second tutorial on tensorflow6. Practical case discussed in details7. Tips for TensorFlow implementation8. Classification and regression with such networks and how the output layer is different9. Softmax for multi-class classification10. Binary classificationChapter 4:  Neural networks error analysisChapter Goal: Describe the problem of identifying the sources of errors (variance, bias, data skewed, not enough data, overfitting, etc.)No of pages:  50-70Sub -Topics1. Train, dev and test dataset – why do we need three? Do we need four? What can we detect with different datasets and how to use them or size them?2. Sources of errors (overfitting, bias, variance, etc.)3. What is overfitting, a discussion4. Why is overfitting important with neural networks?5. Practical case discussion6. A guide on how to perform error analysis7. A practical example with a complete error analysis8. The problem of different datasets (train, dev, test, etc.) coming from different distributions9. Data augmentation techniques and examples10. How to deal with too few data11. How to split the datasets (train, dev, test)? Not 60/20/20 but more 98/1/1 when we have a LOT of data.12. Tips for TensorFlow implementationChapter 5:  Dropout techniqueChapter Goal: Describe what dropout is, when to employ itNo of pages: 30-50Sub -Topics1. What is dropout ?2. When we need to employ dropout3. Different in usage for dropout between training and test set4. How to optimize the dropout parameters5. Tensorflow implementation6. A practical case discussed7. Tips for TensorFlow implementationChapter 6:  Hyper parameters tuningChapter Goal: explain what hyper parameters are, which one are usually tuned, and what it means “hyper parameters optimization”No of pages: 30-50Sub -Topics1. What are hyper parameters2. What are the usually tuned hyper parameters in a deep learning ML project3. How to setup in TensorFlow a ML project so that this optimization is easy4. Practical tips 5. Visualization tips for hyper parameter optimization6. Tips for TensorFlow implementationChapter 7:  Tensorflow and optimizers (Gradient descent, Adam, momentum, etc.)Chapter Goal: Analyze the problem of optimizers and their implementation in tensorflowNo of pages: 50-60Sub -Topics1. Overview of the different optimisation algorithms (Gradient descent, Adam, momentum, etc.) (also mathematically)2. Speed of convergence of the different algorithms3. Hyper parameters that determine the behavior of those optimizer4. Which of those hyper parameters needs tuning?5. Comparison of performance of the different algorithm6. Learning rate dynamical adaptation strategies7. Practical examples8. Tips for TensorFlow implementationChapter 8:  Convolutional Networks and image recognitionChapter Goal: Give the readers a good basis on convolutional networks and how to implement them in tensorflowNo of pages: 30-50Sub -Topics1. What is a convolutional network2. When to use them3. How to develop them with tensorflow4. Practical case explained in detail5. Tips for TensorFlow implementationChapter 9:  Recurrent Neural NetworksChapter Goal: Give the readers a good basis on Recurrent neural networks and how to implement them in tensorflowNo of pages: 30-50Sub -Topics1. What is a RNN2. When to use them3. How to develop them with tensorflow4. Practical case explained in detail5. Tips for TensorFlow implementationChapter 10:  A practical COMPLETE example from scratch (put everything together)Chapter Goal: in this chapter I will put together all that was explained before and do a real-life example ML project (with all aspects included)No of pages: 30-50Sub -Topics1. Discussion of data set (not a simple dataset, something that have real deep-learning potential)2. Clean-up and preparation of data set3. Complete code implementation4. Results analysis and discussion5. Error analysis6. Conclusions7. Tips for TensorFlow implementationChapter 11:  Logistic regression implement from scratch in TensorFlow without librariesChapter Goal: Give the readers a sense of the complexity of implementing a simple method completely from scratch to let them understand how easy is to work with tensorflowNo of pages: 20-30Sub -Topics1. Complete implementation of logistic regression in TensorFlow from scratch and analysis of the code2. Practical example3. Comparison of implementation with sklearn and tensorflow4. Tips for TensorFlow implementation