Data Science from Scratch

Joel Grus is Chief Scientist and Head of Analytics at VoloMetrix, an enterprise analytics startup in Seattle. He has been doing data science since before it was called that, at a handful of Seattle startups before this one, and at Bing when one of those startups got bought by Microsoft. He's been using Python for almost 10 years. Joel is active in the Seattle data science community and maintains a blog at joelgrus.com

Chapter 1Introduction
The Ascendance of Data
What Is Data Science?
Motivating Hypothetical: DataSciencester
Chapter 2A Crash Course in Python
The Basics
The Not-So-Basics
For Further Exploration
Chapter 3Visualizing Data
matplotlib
Bar Charts
Line Charts
Scatterplots
For Further Exploration
Chapter 4Linear Algebra
Vectors
Matrices
For Further Exploration
Chapter 5Statistics
Describing a Single Set of Data
Correlation
Simpson’s Paradox
Some Other Correlational Caveats
Correlation and Causation
For Further Exploration
Chapter 6Probability
Dependence and Independence
Conditional Probability
Bayes’s Theorem
Random Variables
Continuous Distributions
The Normal Distribution
The Central Limit Theorem
For Further Exploration
Chapter 7Hypothesis and Inference
Statistical Hypothesis Testing
Example: Flipping a Coin
Confidence Intervals
P-hacking
Example: Running an A/B Test
Bayesian Inference
For Further Exploration
Chapter 8Gradient Descent
The Idea Behind Gradient Descent
Estimating the Gradient
Using the Gradient
Choosing the Right Step Size
Putting It All Together
Stochastic Gradient Descent
For Further Exploration
Chapter 9Getting Data
stdin and stdout
Reading Files
Scraping the Web
Using APIs
Example: Using the Twitter APIs
For Further Exploration
Chapter 10Working with Data
Exploring Your Data
Cleaning and Munging
Manipulating Data
Rescaling
Dimensionality Reduction
For Further Exploration
Chapter 11Machine Learning
Modeling
What Is Machine Learning?
Overfitting and Underfitting
Correctness
The Bias-Variance Trade-off
Feature Extraction and Selection
For Further Exploration
Chapter 12k-Nearest Neighbors
The Model
Example: Favorite Languages
The Curse of Dimensionality
For Further Exploration
Chapter 13Naive Bayes
A Really Dumb Spam Filter
A More Sophisticated Spam Filter
Implementation
Testing Our Model
For Further Exploration
Chapter 14Simple Linear Regression
The Model
Using Gradient Descent
Maximum Likelihood Estimation
For Further Exploration
Chapter 15Multiple Regression
The Model
Further Assumptions of the Least Squares Model
Fitting the Model
Interpreting the Model
Goodness of Fit
Digression: The Bootstrap
Standard Errors of Regression Coefficients
Regularization
For Further Exploration
Chapter 16Logistic Regression
The Problem
The Logistic Function
Applying the Model
Goodness of Fit
Support Vector Machines
For Further Investigation
Chapter 17Decision Trees
What Is a Decision Tree?
Entropy
The Entropy of a Partition
Creating a Decision Tree
Putting It All Together
Random Forests
For Further Exploration
Chapter 18Neural Networks
Perceptrons
Feed-Forward Neural Networks
Backpropagation
Example: Defeating a CAPTCHA
For Further Exploration
Chapter 19Clustering
The Idea
The Model
Example: Meetups
Choosing k
Example: Clustering Colors
Bottom-up Hierarchical Clustering
For Further Exploration
Chapter 20Natural Language Processing
Word Clouds
n-gram Models
Grammars
An Aside: Gibbs Sampling
Topic Modeling
For Further Exploration
Chapter 21Network Analysis
Betweenness Centrality
Eigenvector Centrality
Directed Graphs and PageRank
For Further Exploration
Chapter 22Recommender Systems
Manual Curation
Recommending What’s Popular
User-Based Collaborative Filtering
Item-Based Collaborative Filtering
For Further Exploration
Chapter 23Databases and SQL
CREATE TABLE and INSERT
UPDATE
DELETE
SELECT
GROUP BY
ORDER BY
JOIN
Subqueries
Indexes
Query Optimization
NoSQL
For Further Exploration
Chapter 24MapReduce
Example: Word Count
Why MapReduce?
MapReduce More Generally
Example: Analyzing Status Updates
Example: Matrix Multiplication
An Aside: Combiners
For Further Exploration
Chapter 25Go Forth and Do Data Science
IPython
Mathematics
Not from Scratch
Find Data
Do Data Science