Network Security with NetFlow and IPFIX

Omar Santos is a Principal Engineer in the Cisco Product Security Incident Response Team (PSIRT) part of Cisco’s Security Research and Operations. He mentors and leads engineers and incident managers during the investigation and resolution of security vulnerabilities in all Cisco products. Omar has been working with information technology and cyber security since the mid-1990s. Omar has designed, implemented, and supported numerous secure networks for Fortune 100 and 500 companies and for the U.S. government. Prior to his current role, he was a Technical Leader within the World Wide Security Practice and the Cisco Technical Assistance Center (TAC), where he taught, led, and mentored many engineers within both organizations. Omar is an active member of the security community, where he leads several industrywide initiatives and standard bodies. His active role helps businesses, academic institutions, state and local law enforcement agencies, and other participants that are dedicated to increasing the security of the critical infrastructure. Omar is the author of several books and numerous whitepapers, articles, and security configuration guidelines and best practices. He has also delivered numerous technical presentations at many conferences and to Cisco customers and partners, in addition to many C-level executive presentations to many organizations. Omar is the author of the following Cisco Press books: CCNA Security 210-260 Official Cert Guide, ISBN-13: 9781587205668 Deploying Next-Generation Firewalls Live Lessons, ISBN-13: 9781587205705 Cisco’s Advanced Malware Protection (AMP), ISBN-13: 9781587144462 Cisco ASA Next-Generation Firewall, IPS, and VPN Services (3rd Edition), ISBN-10: 1587143070 Cisco ASA: All-in-One Firewall, IPS, Anti-X, and VPN Adaptive Security Appliance (2nd Edition), ISBN-10: 1587058197 Cisco ASA: All-in-One Firewall, IPS, and VPN Adaptive Security Appliance, ISBN-10: 1587052091 Cisco Network Admission Control, Volume: Deployment and Management, ISBN-10: 1587052253 End-to-End Network Security: Defense-in-Depth, ISBN-10: 1587053322

Introduction xvi

Chapter 1 Introduction to NetFlow and IPFIX 1

Introduction to NetFlow 1

The Attack Continuum 2

The Network as a Sensor and as an Enforcer 3

What Is a Flow? 4

NetFlow Versus IP Accounting and Billing 6

NetFlow for Network Security 7

Anomaly Detection and DDoS Attacks 8

Data Leak Detection and Prevention 9

Incident Response and Network Security Forensics 9

Traffic Engineering and Network Planning 14

IP Flow Information Export 15

IPFIX Architecture 16

IPFIX Mediators 17

IPFIX Templates 17

Option Templates 19

Introduction to the Stream Control Transmission Protocol (SCTP) 19

Supported Platforms 20

Introduction to Cisco Cyber Threat Defense 21

Cisco Application Visibility and Control and NetFlow 22

Application Recognition 22

Metrics Collection and Exporting 23

Management and Reporting Systems 23

Control 23

Deployment Scenarios 24

Deployment Scenario: User Access Layer 24

Deployment Scenario: Wireless LAN 25

Deployment Scenario: Internet Edge 26

Deployment Scenario: Data Center 28

Public, Private, and Hybrid Cloud Environments 32

Deployment Scenario: NetFlow in Site-to-Site and Remote VPNs 33

NetFlow Remote-Access VPNs 33

NetFlow Site-to-Site VPNs 34

NetFlow Collection Considerations and Best Practices 35

Determining the Flows per Second and Scalability 36

Summary 37

Chapter 2 Cisco NetFlow Versions and Features 39

NetFlow Versions and Respective Features 39

NetFlow v1 Flow Header Format and Flow Record Format 40

NetFlow v5 Flow Header Format and Flow Record Format 41

NetFlow v7 Flow Header Format and Flow Record Format 42

NetFlow Version 9 43

NetFlow and IPFIX Comparison 57

Summary 57

Chapter 3 Cisco Flexible NetFlow 59

Introduction to Cisco’s Flexible NetFlow 59

Simultaneous Application Tracking 60

Flexible NetFlow Records 61

Flexible NetFlow Key Fields 61

Flexible NetFlow Non-Key Fields 63

NetFlow Predefined Records 65

User-Defined Records 65

Flow Monitors 65

Flow Exporters 65

Flow Samplers 66

Flexible NetFlow Configuration 66

Configure a Flow Record 67

Configuring a Flow Monitor for IPv4 or IPv6 69

Configuring a Flow Exporter for the Flow Monitor 71

Applying a Flow Monitor to an Interface 73

Flexible NetFlow IPFIX Export Format 74

Summary 74

Chapter 4 NetFlow Commercial and Open Source Monitoring and Analysis Software Packages 75

Commercial NetFlow Monitoring and Analysis Software Packages 75

Lancope’s StealthWatch Solution 76

Plixer’s Scrutinizer 79

Open Source NetFlow Monitoring and Analysis Software Packages 80

NFdump 81

NfSen 86

SiLK 86

SiLK Configuration Files 87

Filtering, Displaying, and Sorting NetFlow Records with SiLK 87

SiLK’s Python Extension 88

Counting, Grouping, and Mating NetFlow Records with Silk 88

SiLK IPset, Bag, and Prefix Map Manipulation Tools 88

IP and Port Labeling Files 89

SiLK Runtime Plug-Ins 89

SiLK Utilities for Packet Capture and IPFIX Processing 90

Utilities to Detect Network Scans 90

SiLK Flow File Utilities 90

Additional SiLK Utilities 91

Elasticsearch, Logstash, and Kibana Stack 92

Elasticsearch 92

Logstash 92

Kibana 93

Elasticsearch Marvel and Shield 94

ELK Deployment Topology 94

Installing ELK 95

Installing Elasticsearch 96

Install Kibana 105

Installing Nginx 106

Install Logstash 107

Summary 109

Chapter 5 Big Data Analytics and NetFlow 111

Introduction to Big Data Analytics for Cyber Security 111

What Is Big Data? 111

Unstructured Versus Structured Data 112

Extracting Value from Big Data 113

NetFlow and Other Telemetry Sources for Big Data Analytics for Cyber Security 114

OpenSOC 115

Hadoop 116

HDFS 117

Flume 119

Kafka 120

Storm 121

Hive 122

Elasticsearch 123

HBase 124

Third-Party Analytic Tools 125

Other Big Data Projects in the Industry 126

Understanding Big Data Scalability: Big Data Analytics in the Internet of Everything 127

Summary 128

Chapter 6 Cisco Cyber Threat Defense and NetFlow 129

Overview of the Cisco Cyber Threat Defense Solution 129

The Attack Continuum 130

Cisco CTD Solution Components 131

NetFlow Platform Support 133

Traditional NetFlow Support in Cisco IOS Software 133

NetFlow Support in Cisco IOS-XR Software 135

Flexible NetFlow Support 135

NetFlow Support in Cisco ASA 140

Deploying the Lancope StealthWatch System 140

Deploying StealthWatch FlowCollectors 142

StealthWatch FlowReplicators 146

StealthWatch Management Console 146

Deploying NetFlow Secure Event Logging in the Cisco ASA 148

Deploying NSEL in Cisco ASA Configured for Clustering 151

Unit Roles and Functions in Clustering 152

Clustering NSEL Operations 152

Configuring NSEL in the Cisco ASA 153

Configuring NSEL in the Cisco ASA Using ASDM 153

Configuring NSEL in the Cisco ASA Using the CLI 155

NSEL and Syslog 156

Defining the NSEL Export Policy 157

Monitoring NSEL 159

Configuring NetFlow in the Cisco Nexus 1000V 160

Defining a Flow Record 161

Defining the Flow Exporter 162

Defining a Flow Monitor 163

Applying the Flow Monitor to an Interface 164

Configuring NetFlow in the Cisco Nexus 7000 Series 164

Configuring the Cisco NetFlow Generation Appliance 166

Initializing the Cisco NGA 166

Configuring NetFlow in the Cisco NGA via the GUI 168

Configuring NetFlow in the Cisco NGA via the CLI 169

Additional Cisco CTD Solution Components 171

Cisco ASA 5500-X Series Next-Generation Firewalls and the Cisco ASA with FirePOWER Services 171

Next-Generation Intrusion Prevention Systems 172

FireSIGHT Management Center 173

AMP for Endpoints 173

AMP for Networks 176

AMP Threat Grid 176

Email Security 177

Email Security Appliance 177

Cloud Email Security 179

Cisco Hybrid Email Security 179

Web Security 180

Web Security Appliance 180

Cisco Content Security Management Appliance 184

Cisco Cloud Web Security 185

Cisco Identity Services Engine 186

Summary 187

Chapter 7 Troubleshooting NetFlow 189

Troubleshooting Utilities and Debug Commands 189

Troubleshooting NetFlow in Cisco IOS and Cisco IOS XE Devices 194

Cisco IOS Router Flexible NetFlow Configuration 195

Troubleshooting Communication Problems with the NetFlow Collector 201

Additional Useful Troubleshooting Debug and Show Commands 204

Verifying a Flow Monitor Configuration 204

Displaying Flow Exporter Templates and Export IDs 207

Debugging Flow Records 212

Preventing Export Storms with Flexible NetFlow 213

Troubleshooting NetFlow in Cisco NX-OS Software 214

Troubleshooting NetFlow in Cisco IOS-XR Software 217

Flow Exporter Statistics and Diagnostics 219

Flow Monitor Statistics and Diagnostics 222

Displaying NetFlow Producer Statistics in Cisco IOS-XR 226

Additional Useful Cisco IOS-XR Show Commands 228

Troubleshooting NetFlow in the Cisco ASA 228

Troubleshooting NetFlow in the Cisco NetFlow Generation Appliance 235

Gathering Information About Configured NGA Managed Devices 235

Gathering Information About the Flow Collector 236

Gathering Information About the Flow Exporter 237

Gathering Information About Flow Records 237

Gathering Information About the Flow Monitor 238

Show Tech-Support 239

Additional Useful NGA show Commands 245

Summary 246

Chapter 8 Case Studies 247

Using NetFlow for Anomaly Detection and Identifying DoS Attacks 247

Direct DDoS Attacks 248

Reflected DDoS Attacks 248

Amplification Attacks 249

Identifying DDoS Attacks Using NetFlow 250

Using NetFlow in Enterprise Networks to Detect DDoS Attacks 250

Using NetFlow in Service Provider Networks to Detect DDoS Attacks 253

Using NetFlow for Incident Response and Forensics 254

Credit Card Theft 254

Theft of Intellectual Property 259

Using NetFlow for Monitoring Guest Users and Contractors 262

Using NetFlow for Capacity Planning 267

Using NetFlow to Monitor Cloud Usage 269

Summary 271

TOC, 9781587144387, 8/25/2015