Data Analytics for IT Networks

John Garrett is CCIE Emeritus (6204) and Splunk Certified. He earned an M.S. in predictive analytics from Northwestern University, and has a patent pending related to analysis of network devices with data science techniques. John has architected, designed, and implemented LAN, WAN, wireless, and data center solutions for some of the largest Cisco customers. As a secondary role, John has worked with teams in the Cisco Services organization to innovate on some of the most widely used tools and methodologies at Customer Experience over the past 12 years. For the past 7 years, John’s journey has moved through server virtualization, network virtualization, OpenStack and cloud, network functions virtualization (NFV), service assurance, and data science. The realization that analytics and data science play roles in all these brought John full circle back to developing innovative tools and techniques for Cisco Services. John’s most recent role is as an Analytics Technical Lead, developing use cases to benefit Cisco Services customers as part of Business Critical Services for Cisco. John lives with his wife and children in Raleigh, North Carolina.

Foreword xvii
Introduction: Your future is in your hands! xviii
Chapter 1 Getting Started with Analytics 1
What This Chapter Covers 1
Data: You as the SME 2
Use-Case Development with Bias and Mental Models 2
Data Science: Algorithms and Their Purposes 3
What This Book Does Not Cover 4
Building a Big Data Architecture 4
Microservices Architectures and Open Source Software 5
R Versus Python Versus SAS Versus Stata 6
Databases and Data Storage 6
Cisco Products in Detail 6
Analytics and Literary Perspectives 7
Analytics Maturity 7
Knowledge Management 8
Gartner Analytics 8
Strategic Thinking 9
Striving for “Up and to the Right” 9
Moving Your Perspective 10
Hot Topics in the Literature 11
Summary 12
Chapter 2 Approaches for Analytics and Data Science 13
Model Building and Model Deployment 14
Analytics Methodology and Approach 15
Common Approach Walkthrough 16
Distinction Between the Use Case and the Solution 18
Logical Models for Data Science and Data 19
Analytics as an Overlay 20
Analytics Infrastructure Model 22
Summary 33
Chapter 3 Understanding Networking Data Sources 35
Planes of Operation on IT Networks 36
Review of the Planes 40
Data and the Planes of Operation 42
Planes Data Examples 44
A Wider Rabbit Hole 49
A Deeper Rabbit Hole 51
Summary 53
Chapter 4 Accessing Data from Network Components 55
Methods of Networking Data Access 55
Pull Data Availability 57
Push Data Availability 61
Control Plane Data 67
Data Plane Traffic Capture 68
Packet Data 70
Other Data Access Methods 74
Data Types and Measurement Considerations 76
Numbers and Text 77
Data Structure 82
Data Manipulation 84
Other Data Considerations 87
External Data for Context 89
Data Transport Methods 89
Transport Considerations for Network Data Sources 90
Summary 96
Chapter 5 Mental Models and Cognitive Bias 97
Changing How You Think 98
Domain Expertise, Mental Models, and Intuition 99
Mental Models 99
Daniel Kahneman’s System 1 and System 2 102
Intuition 103
Opening Your Mind to Cognitive Bias 104
Changing Perspective, Using Bias for Good 105
Your Bias and Your Solutions 106
How You Think: Anchoring, Focalism, Narrative Fallacy, Framing, and Priming 107
How Others Think: Mirroring 110
What Just Happened? Availability, Recency, Correlation, Clustering, and Illusion of Truth 111
Enter the Boss: HIPPO and Authority Bias 113
What You Know: Confirmation, Expectation, Ambiguity, Context, and Frequency Illusion 114
What You Don’t Know: Base Rates, Small Numbers, Group Attribution, and Survivorship 117
Your Skills and Expertise: Curse of Knowledge, Group Bias, and Dunning-Kruger 119
We Don’t Need a New System: IKEA, Not Invented Here, Pro-Innovation, Endowment, Status Quo, Sunk Cost, Zero Price, and Empathy 121
I Knew It Would Happen: Hindsight, Halo Effect, and Outcome Bias 123
Summary 124
Chapter 6 Innovative Thinking Techniques 127
Acting Like an Innovator and Mindfulness 128
Innovation Tips and Techniques 129
Developing Analytics for Your Company 140
Defocusing, Breaking Anchors, and Unpriming 140
Lean Thinking 142
Cognitive Trickery 143
Quick Innovation Wins 143
Summary 144
Chapter 7 Analytics Use Cases and the Intuition Behind Them 147
Analytics Definitions 150
How to Use the Information from This Chapter 151
Priming and Framing Effects 151
Analytics Rube Goldberg Machines 151
Popular Analytics Use Cases 152
Machine Learning and Statistics Use Cases 153
Common IT Analytics Use Cases 170
Broadly Applicable Use Cases 199
Some Final Notes on Use Cases 214
Summary 214
Chapter 8 Analytics Algorithms and the Intuition Behind Them 217
About the Algorithms 217
Algorithms and Assumptions 218
Additional Background 219
Data and Statistics 221
Statistics 221
Correlation 224
Longitudinal Data 225
ANOVA 227
Probability 228
Bayes’ Theorem 228
Feature Selection 230
Data-Encoding Methods 232
Dimensionality Reduction 233
Unsupervised Learning 234
Clustering 234
Association Rules 240
Sequential Pattern Mining 243
Collaborative Filtering 244
Supervised Learning 246
Regression Analysis 246
Classification Algorithms 248
Decision Trees 249
Random Forest 250
Gradient Boosting Methods 251
Neural Networks 252
Support Vector Machines 258
Time Series Analysis 259
Text and Document Analysis 262
Natural Language Processing (NLP) 262
Information Retrieval 263
Topic Modeling 265
Sentiment Analysis 266
Other Analytics Concepts 267
Artificial Intelligence 267
Confusion Matrix and Contingency Tables 267
Cumulative Gains and Lift 269
Simulation 271
Summary 271
Chapter 9 Building Analytics Use Cases 273
Designing Your Analytics Solutions 274
Using the Analytics Infrastructure Model 275
About the Upcoming Use Cases 276
The Data 276
The Data Science 278
The Code 280
Operationalizing Solutions as Use Cases 281
Understanding and Designing Workflows 282
Tips for Setting Up an Environment to Do Your Own Analysis 282
Summary 284
Chapter 10 Developing Real Use Cases: The Power of Statistics 285
Loading and Exploring Data 286
Base Rate Statistics for Platform Crashes 288
Base Rate Statistics for Software Crashes 299
ANOVA 305
Data Transformation 310
Tests for Normality 311
Examining Variance 313
Statistical Anomaly Detection 318
Summary 321
Chapter 11 Developing Real Use Cases: Network Infrastructure Analytics 323
Human DNA and Fingerprinting 324
Building Search Capability 325
Loading Data and Setting Up the Environment 325
Encoding Data for Algorithmic Use 328
Search Challenges and Solutions 331
Other Uses of Encoded Data 336
Dimensionality Reduction 337
Data Visualization 340
K-Means Clustering 344
Machine Learning Guided Troubleshooting 350
Summary 353
Chapter 12 Developing Real Use Cases: Control Plane Analytics Using Syslog Telemetry 355
Data for This Chapter 356
OSPF Routing Protocols 357
Non-Machine Learning Log Analysis Using pandas 357
Noise Reduction 360
Finding the Hotspots 362
Machine Learning—Based Log Evaluation 366
Data Visualization 367
Cleaning and Encoding Data 369
Clustering 373
More Data Visualization 375
Transaction Analysis 379
Task List 386
Summary 387
Chapter 13 Developing Real Use Cases: Data Plane Analytics 389
The Data 390
SME Analysis 394
SME Port Clustering 407
Machine Learning: Creating Full Port Profiles 413
Machine Learning: Creating Source Port Profiles 419
Asset Discovery 422
Investigation Task List 423
Summary 424
Chapter 14 Cisco Analytics 425
Architecture and Advisory Services for Analytics 426
Stealthwatch 427
Digital Network Architecture (DNA) 428
AppDynamics 428
Tetration 430
Crosswork Automation 431
IoT Analytics 432
Analytics Platforms and Partnerships 433
Cisco Open Source Platform 433
Summary 434
Chapter 15 Book Summary 435
Analytics Introduction and Methodology 436
All About Networking Data 438
Using Bias and Innovation to Discover Solutions 439
Analytics Use Cases and Algorithms 439
Building Real Analytics Use Cases 440
Cisco Services and Solutions 442
In Closing 442
Appendix A Function for Parsing Packets from pcap Files 443
9781587145131, TOC, 9/19/18