Communication Networks and Service Management in the Era of Artificial Intelligence and Machine Learning

Nur Zincir-Heywood, PhD, is Full Professor of Computer Science with Dalhousie University in Nova Scotia, Canada. She is an Associate Editor of the IEEE Transactions on Network and Service Management and Wiley International Journal of Network Management. Marco Mellia, PhD, is Full Professor with Politecnico di Torino, Italy. He is an Associate Editor of the IEEE Transactions on Network and Service Management, Elsevier Computer Networks and ACM Computer Communication Reviews. Yixin Diao, PhD, is Director of Data Science and Analytics at PebblePost in New York, NY, USA. He is an Associate Editor of the IEEE Transactions on Network and Service Management and the Journal of Network and Systems Management.

List of Contributors xv

Preface xxi

Acknowledgments xxv

Acronyms xxvii

Part I Introduction 1

1 Overview of Network and Service Management 3
Marco Mellia, Nur Zincir-Heywood, and Yixin Diao

1.1 Network and Service Management at Large 3

1.2 Data Collection and Monitoring Protocols 5

1.2.1 SNMP Protocol Family 5

1.2.2 Syslog Protocol 5

1.2.3 IP Flow Information eXport (IPFIX) 6

1.2.4 IP Performance Metrics (IPPM) 7

1.2.5 Routing Protocols and Monitoring Platforms 8

1.3 Network Configuration Protocol 9

1.3.1 Standard Configuration Protocols and Approaches 9

1.3.2 Proprietary Configuration Protocols 10

1.3.3 Integrated Platforms for Network Monitoring 10

1.4 Novel Solutions and Scenarios 12

1.4.1 Software-Defined Networking – SDN 12

1.4.2 Network Functions Virtualization –NFV 14

Bibliography 15

2 Overview of Artificial Intelligence and Machine Learning 19
Nur Zincir-Heywood, Marco Mellia, and Yixin Diao

2.1 Overview 19

2.2 Learning Algorithms 20

2.2.1 Supervised Learning 21

2.2.2 Unsupervised Learning 22

2.2.3 Reinforcement Learning 23

2.3 Learning for Network and Service Management 24

Bibliography 26

Part II Management Models and Frameworks 33

3 Managing Virtualized Networks and Services with Machine Learning 35
Raouf Boutaba, Nashid Shahriar, Mohammad A. Salahuddin, and Noura Limam

3.1 Introduction 35

3.2 Technology Overview 37

3.2.1 Virtualization of Network Functions 38

3.2.1.1 Resource Partitioning 38

3.2.1.2 Virtualized Network Functions 40

3.2.2 Link Virtualization 41

3.2.2.1 Physical Layer Partitioning 41

3.2.2.2 Virtualization at Higher Layers 42

3.2.3 Network Virtualization 42

3.2.4 Network Slicing 43

3.2.5 Management and Orchestration 44

3.3 State-of-the-Art 46

3.3.1 Network Virtualization 46

3.3.2 Network Functions Virtualization 49

3.3.2.1 Placement 49

3.3.2.2 Scaling 52

3.3.3 Network Slicing 55

3.3.3.1 Admission Control 55

3.3.3.2 Resource Allocation 56

3.4 Conclusion and Future Direction 59

3.4.1 Intelligent Monitoring 60

3.4.2 Seamless Operation and Maintenance 60

3.4.3 Dynamic Slice Orchestration 61

3.4.4 Automated Failure Management 61

3.4.5 Adaptation and Consolidation of Resources 61

3.4.6 Sensitivity to Heterogeneous Hardware 62

3.4.7 Securing Machine Learning 62

Bibliography 63

4 Self-Managed 5G Networks 69
Jorge Martín-Pérez, Lina Magoula, Kiril Antevski, Carlos Guimarães, Jorge Baranda, Carla Fabiana Chiasserini, Andrea Sgambelluri, Chrysa Papagianni, Andrés García-Saavedra, Ricardo Martínez, Francesco Paolucci, Sokratis Barmpounakis, Luca Valcarenghi, Claudio EttoreCasetti, Xi Li, Carlos J. Bernardos, Danny De Vleeschauwer, Koen De Schepper, Panagiotis Kontopoulos, Nikolaos Koursioumpas, Corrado Puligheddu, Josep Mangues-Bafalluy, and Engin Zeydan

4.1 Introduction 69

4.2 Technology Overview 73

4.2.1 RAN Virtualization and Management 73

4.2.2 Network Function Virtualization 75

4.2.3 Data Plane Programmability 76

4.2.4 Programmable Optical Switches 77

4.2.5 Network Data Management 78

4.3 5G Management State-of-the-Art 80

4.3.1 RAN resource management 80

4.3.1.1 Context-Based Clustering and Profiling for User and Network Devices 80

4.3.1.2 Q-Learning Based RAN Resource Allocation 81

4.3.1.3 vrAIn: AI-Assisted Resource Orchestration for Virtualized Radio Access Networks 81

4.3.2 Service Orchestration 83

4.3.3 Data Plane Slicing and Programmable Traffic Management 85

4.3.4 Wavelength Allocation 86

4.3.5 Federation 88

4.4 Conclusions and Future Directions 89

Bibliography 92

5 AI in 5G Networks: Challenges and Use Cases 101
Stanislav Lange, Susanna Schwarzmann, Marija Gaji´c, Thomas Zinner, and Frank A. Kraemer

5.1 Introduction 101

5.2 Background 103

5.2.1 ML in the Networking Context 103

5.2.2 ML in Virtualized Networks 104

5.2.3 ML for QoE Assessment and Management 104

5.3 Case Studies 105

5.3.1 QoE Estimation and Management 106

5.3.1.1 Main Challenges 107

5.3.1.2 Methodology 108

5.3.1.3 Results and Guidelines 109

5.3.2 Proactive VNF Deployment 110

5.3.2.1 Problem Statement and Main Challenges 111

5.3.2.2 Methodology 112

5.3.2.3 Evaluation Results and Guidelines 113

5.3.3 Multi-service, Multi-domain Interconnect 115

5.4 Conclusions and Future Directions 117

Bibliography 118

6 Machine Learning for Resource Allocation in Mobile Broadband Networks 123
Sadeq B. Melhem, Arjun Kaushik, Hina Tabassum, and Uyen T. Nguyen

6.1 Introduction 123

6.2 ML in Wireless Networks 124

6.2.1 Supervised ML 124

6.2.1.1 Classification Techniques 125

6.2.1.2 Regression Techniques 125

6.2.2 Unsupervised ML 126

6.2.2.1 Clustering Techniques 126

6.2.2.2 Soft Clustering Techniques 127

6.2.3 Reinforcement Learning 127

6.2.4 Deep Learning 128

6.2.5 Summary 129

6.3 ML-Enabled Resource Allocation 129

6.3.1 Power Control 131

6.3.1.1 Overview 131

6.3.1.2 State-of-the-Art 131

6.3.1.3 Lessons Learnt 132

6.3.2 Scheduling 132

6.3.2.1 Overview 132

6.3.2.2 State-of-the-Art 132

6.3.2.3 Lessons Learnt 134

6.3.3 User Association 134

6.3.3.1 Overview 134

6.3.3.2 State-of-the-Art 136

6.3.3.3 Lessons Learnt 136

6.3.4 Spectrum Allocation 136

6.3.4.1 Overview 136

6.3.4.2 State-of-the-Art 138

6.3.4.3 Lessons Learnt 138

6.4 Conclusion and Future Directions 140

6.4.1 Transfer Learning 140

6.4.2 Imitation Learning 140

6.4.3 Federated-Edge Learning 141

6.4.4 Quantum Machine Learning 142

Bibliography 142

7 Reinforcement Learning for Service Function Chain Allocation in Fog Computing 147
José Santos, Tim Wauters, Bruno Volckaert, and Filip De Turck

7.1 Introduction 147

7.2 Technology Overview 148

7.2.1 Fog Computing (FC) 149

7.2.2 Resource Provisioning 149

7.2.3 Service Function Chaining (SFC) 150

7.2.4 Micro-service Architecture 150

7.2.5 Reinforcement Learning (RL) 151

7.3 State-of-the-Art 152

7.3.1 Resource Allocation for Fog Computing 152

7.3.2 ML Techniques for Resource Allocation 153

7.3.3 RL Methods for Resource Allocation 154

7.4 A RL Approach for SFC Allocation in Fog Computing 155

7.4.1 Problem Formulation 155

7.4.2 Observation Space 156

7.4.3 Action Space 157

7.4.4 Reward Function 158

7.4.5 Agent 161

7.5 Evaluation Setup 162

7.5.1 Fog–Cloud Infrastructure 162

7.5.2 Environment Implementation 162

7.5.3 Environment Configuration 164

7.6 Results 165

7.6.1 Static Scenario 165

7.6.2 Dynamic Scenario 167

7.7 Conclusion and Future Direction 169

Bibliography 170

Part III Management Functions and Applications 175

8 Designing Algorithms for Data-Driven Network Management and Control: State-of-the-Art and Challenges 177
Andreas Blenk, Patrick Kalmbach, Johannes Zerwas, and Stefan Schmid

8.1 Introduction 177

8.1.1 Contributions 179

8.1.2 Exemplary Network Use Case Study 179

8.2 Technology Overview 181

8.2.1 Data-Driven Network Optimization 181

8.2.2 Optimization Problems over Graphs 182

8.2.3 From Graphs to ML/AI Input 184

8.2.4 End-to-End Learning 187

8.3 Data-Driven Algorithm Design: State-of-the Art 188

8.3.1 Data-Driven Optimization in General 188

8.3.2 Data-Driven Network Optimization 190

8.3.3 Non-graph Related Problems 192

8.4 Future Direction 193

8.4.1 Data Production and Collection 193

8.4.2 ML and AI Advanced Algorithms for Network Management with Performance Guarantees 194

8.5 Summary 194

Acknowledgments 195

Bibliography 195

9 AI-Driven Performance Management in Data-Intensive Applications 199
Ahmad Alnafessah, Gabriele Russo Russo, Valeria Cardellini, Giuliano Casale, and Francesco Lo Presti

9.1 Introduction 199

9.2 Data-Processing Frameworks 200

9.2.1 Apache Storm 200

9.2.2 Hadoop MapReduce 201

9.2.3 Apache Spark 202

9.2.4 Apache Flink 202

9.3 State-of-the-Art 203

9.3.1 Optimal Configuration 203

9.3.1.1 Traditional Approaches 203

9.3.1.2 AI Approaches 204

9.3.1.3 Example: AI-Based Optimal Configuration 206

9.3.2 Performance Anomaly Detection 207

9.3.2.1 Traditional Approaches 208

9.3.2.2 AI Approaches 208

9.3.2.3 Example: ANNs-Based Anomaly Detection 210

9.3.3 Load Prediction 211

9.3.3.1 Traditional Approaches 212

9.3.3.2 AI Approaches 212

9.3.4 Scaling Techniques 213

9.3.4.1 Traditional Approaches 213

9.3.4.2 AI Approaches 214

9.3.5 Example: RL-Based Auto-scaling Policies 214

9.4 Conclusion and Future Direction 216

Bibliography 217

10 Datacenter Traffic Optimization with Deep Reinforcement Learning 223
Li Chen, Justinas Lingys, Kai Chen, and Xudong Liao

10.1 Introduction 223

10.2 Technology Overview 225

10.2.1 Deep Reinforcement Learning (DRL) 226

10.2.2 Applying ML to Networks 227

10.2.3 Traffic Optimization Approaches in Datacenter 229

10.2.4 Example: DRL for Flow Scheduling 230

10.2.4.1 Flow Scheduling Problem 230

10.2.4.2 DRL Formulation 230

10.2.4.3 DRL Algorithm 231

10.3 State-of-the-Art: AuTO Design 231

10.3.1 Problem Identified 231

10.3.2 Overview 232

10.3.3 Peripheral System 233

10.3.3.1 Enforcement Module 233

10.3.3.2 Monitoring Module 234

10.3.4 Central System 234

10.3.5 DRL Formulations and Solutions 235

10.3.5.1 Optimizing MLFQ Thresholds 235

10.3.5.2 Optimizing Long Flows 239

10.4 Implementation 239

10.4.1 Peripheral System 239

10.4.1.1 Monitoring Module (MM): 240

10.4.1.2 Enforcement Module (EM): 240

10.4.2 Central System 241

10.4.2.1 sRLA 241

10.4.2.2 lRLA 242

10.5 Experimental Results 242

10.5.1 Setting 243

10.5.2 Comparison Targets 244

10.5.3 Experiments 244

10.5.3.1 Homogeneous Traffic 244

10.5.3.2 Spatially Heterogeneous Traffic 245

10.5.3.3 Temporally and Spatially Heterogeneous Traffic 246

10.5.4 Deep Dive 247

10.5.4.1 Optimizing MLFQ Thresholds using DRL 247

10.5.4.2 Optimizing Long Flows using DRL 248

10.5.4.3 System Overhead 249

10.6 Conclusion and Future Directions 251

Bibliography 253

11 The New Abnormal: Network Anomalies in the AI Era 261
Francesca Soro, Thomas Favale, Danilo Giordano, Luca Vassio, Zied Ben Houidi, and Idilio Drago

11.1 Introduction 261

11.2 Definitions and Classic Approaches 262

11.2.1 Definitions 263

11.2.2 Anomaly Detection: A Taxonomy 263

11.2.3 Problem Characteristics 264

11.2.4 Classic Approaches 266

11.3 AI and Anomaly Detection 267

11.3.1 Methodology 267

11.3.2 Deep Neural Networks 268

11.3.3 Representation Learning 270

11.3.4 Autoencoders 271

11.3.5 Generative Adversarial Networks 272

11.3.6 Reinforcement Learning 274

11.3.7 Summary and Takeaways 275

11.4 Technology Overview 277

11.4.1 Production-Ready Tools 277

11.4.2 Research Alternatives 279

11.4.3 Summary and Takeaways 280

11.5 Conclusions and Future Directions 282

Bibliography 283

12 Automated Orchestration of Security Chains Driven by Process Learning 289
Nicolas Schnepf, Rémi Badonnel, Abdelkader Lahmadi, and Stephan Merz

12.1 Introduction 289

12.2 RelatedWork 290

12.2.1 Chains of Security Functions 291

12.2.2 Formal Verification of Networking Policies 292

12.3 Background 294

12.3.1 Flow-Based Detection of Attacks 294

12.3.2 Programming SDN Controllers 295

12.4 Orchestration of Security Chains 296

12.5 Learning Network Interactions 298

12.6 Synthesizing Security Chains 301

12.7 Verifying Correctness of Chains 306

12.7.1 Packet Routing 306

12.7.2 Shadowing Freedom and Consistency 306

12.8 Optimizing Security Chains 308

12.9 Performance Evaluation 311

12.9.1 Complexity of Security Chains 312

12.9.2 Response Times 313

12.9.3 Accuracy of Security Chains 313

12.9.4 Overhead Incurred by Deploying Security Chains 314

12.10 Conclusions 315

Bibliography 316

13 Architectures for Blockchain-IoT Integration 321
Sina Rafati Niya, Eryk Schiller, and Burkhard Stiller

13.1 Introduction 321

13.1.1 Blockchain Basics 323

13.1.2 Internet-of-Things (IoT) Basics 324

13.2 Blockchain-IoT Integration (BIoT) 325

13.2.1 BIoT Potentials 326

13.2.2 BIoT Use Cases 328

13.2.3 BIoT Challenges 329

13.2.3.1 Scalability 332

13.2.3.2 Security 333

13.2.3.3 Energy Efficiency 334

13.2.3.4 Manageability 335

13.3 BIoT Architectures 335

13.3.1 Cloud, Fog, and Edge-Based Architectures 337

13.3.2 Software-Defined Architectures 337

13.3.3 A Potential Standard BIoT Architecture 338

13.4 Summary and Considerations 341

Bibliography 342

Index 345