Web and Network Data Science

THOMAS W. MILLER is faculty director of the Predictive Analytics program at Northwestern University. He has designed courses for the program, including Marketing Analytics, Advanced Modeling Techniques, Data Visualization, Web and Network Data Science, and the capstone course. He has taught extensively in the program and works with more than forty other faculty members in delivering training in predictive analytics and data science.   Miller is co-founder and director of product development at ToutBay, a publisher and distributor of data science applications. He has consulted widely in the areas of retail site selection, product positioning, segmentation, and pricing in competitive markets, and has worked with predictive models for over 30 years. Miller’s books include Modeling Techniques in Predictive Analytics (Revised and Expanded Edition), Modeling Techniques in Predictive Analytics with Python and R, Data and Text Mining: A Business Applications Approach, Research and Information Services: An Integrated Approach for Business, and a book about predictive modeling in sports, Without a Tout: How to Pick a Winning Team.   Before entering academia, Miller spent nearly 15 years in business IT in the computer and transportation industries. He also directed the A. C. Nielsen Center for Marketing Research and taught market research and business strategy at the University of Wisconsin—Madison.   He holds a Ph.D. in psychology (psychometrics) and a master’s degree in statistics from the University of Minnesota, and an MBA and master’s degree in economics from the University of Oregon.  

Preface v

1  Being Technically Inclined 1

2  Delivering a Message Online 13

3  Crawling and Scraping the Web 25

4  Testing Links, Look, and Feel 43

5  Watching Competitors 55

6  Visualizing Networks 69

7  Understanding Communities 95

8  Measuring Sentiment 119

9  Discovering Common Themes 171

10  Making Recommendations 201

11  Playing Network Games 223

12  What’s Next for the Web? 233

A  Data Science Methods 237

A.1  Databases and Data Preparation 240

A.2  Classical and Bayesian Statistics 242

A.3  Regression and Classification 245

A.4  Machine Learning 250

A.5  Data Visualization 252

A.6  Text Analytics 253

B  Primary Research Online 261

C  Case Studies 281

C.1  Email or Spam? 281

C.2  ToutBay Begins 284

C.3  Keyword Games: Dodgers and Angels 288

C.4  Enron Email Corpus and Network 291

C.5  Wikipedia Votes 292

C.6  Quake Talk 294

C.7  POTUS Speeches 295

C.8  Anonymous Microsoft Web Data 296

D  Code and Utilities 297

E  Glossary 313

Bibliography 321

Index 351