Beyond the Algorithm

Omar Santos is a cybersecurity thought leader with a passion for driving industry-wide initiatives to enhance the security of critical infrastructures. Omar is the lead of the DEF CON Red Team Village, the chair of the Common Security Advisory Framework (CSAF) technical committee, the founder of OpenEoX, and board member of the OASIS Open standards organization. Omar’s collaborative efforts extend to numerous organizations, including the Forum of Incident Response and Security Teams (FIRST) and the Industry Consortium for Advancement of Security on the Internet (ICASI).   Omar is a renowned expert in ethical hacking, vulnerability research, incident response, and AI security. He employs his deep understanding of these disciplines to help organizations stay ahead of emerging threats. His dedication to cybersecurity has made a significant impact on businesses, academic institutions, law enforcement agencies, and other entities striving to bolster their security measures.   With more than 20 books, video courses, white papers, and technical articles under his belt, Omar’s expertise is widely recognized and respected. Omar is a Distinguished Engineer at Cisco focusing on AI security, research, incident response, and vulnerability disclosure. You can follow Omar on Twitter @santosomar.   Petar Radanliev is a Postdoctoral Research Associate at the Department of Computer Science at the University of Oxford. He obtained his PhD at the University of Wales in 2014. He continued with postdoctoral research at Imperial College London, the University of Cambridge, Massachusetts Institute of Technology, and the Department of Engineering Science at the University of Oxford before moving to the Department of Computer Science. His current research focuses on artificial intelligence, cybersecurity, quantum computing, and blockchain technology. Before joining academia, Dr. Petar Radanliev spent ten years as a Cybersecurity Manager for RBS, the largest bank in the world at the time, and five years as a Lead Penetration Tester for the Ministry for Defence.

Preface.. . . . . . . . . . . . . . . . . xvii

1 Historical Overview of Artificial Intelligence (AI) and Machine Learning (ML).. . . . . . . . . . . . 1

The Story of Eva. . . . . . . . . . . . . 2

The Origins.. . . . . . . . . . . . . . 3

Advancements of Artificial Intelligence.. . . . . . . . . 5

Understanding AI and ML. . . . . . . . . . . 8

Concluding the Story of Eva.. . . . . . . . . . . 25

Summary. . . . . . . . . . . . . . 26

Test Your Skills. . . . . . . . . . . . . 27

Exercise 1-1: Exploring the Historical Development and Ethical Concerns of AI.. . . . . . . . . . . . 29

Exercise 1-2: Understanding AI and ML. . . . . . . . . 29

Exercise 1-3: Comparison of ML Algorithms.. . . . . . . . 30

Exercise 1-4: Assessing Applications of ML Algorithms. . . . . . 31

2 Fundamentals of AI and ML Technologies and Implementations.. . 33

What Are the Leading AI and ML Technologies and Algorithms?. . . . 34

ChatGPT and the Leading AI and ML Technologies: Exploring

Capabilities and Applications.. . . . . . . . . . 43

Understanding the Two Categories of AI: Capability-Based Types and Functionality-Based Types. . . . . . . . 52

Leveraging AI and ML to Tackle Real-World Challenges: A Case Study. . . 53

Reflecting on the Societal and Ethical Implications of AI Technologies.. . 54

Assessing Future Trends and Emerging Developments in AI and ML Technologies.. . . . . . . . . . . . . 56

Summary. . . . . . . . . . . . . . 57

Test Your Skills. . . . . . . . . . . . . 59

Exercise 2-1: Algorithm Selection Exercise: Matching Scenarios with Appropriate Machine Learning Techniques.. . . . . . . . 61

Exercise 2-2: Exploring AI and ML Technologies.. . . . . . . 63

Exercise 2-3: Capabilities and Benefits of AI-Optimized Hardware.. . . 63

Exercise 2-4: Understanding the Two Categories of AI.. . . . . . 64

Exercise 2-5: Future Trends and Emerging Developments in AI and ML Technologies. . . . . . . . . . . . . . 64

3 Generative AI and Large Language Models.. . . . . . . 65

Introduction to Generative AI and LLMs. . . . . . . . 66

Large Language Models (LLMs): Revolutionizing Natural Language Processing (NLP). . . . . . . . . . . 88

Hugging Face.. . . . . . . . . . . . . 95

Auto-GPT: A Revolutionary Step in Autonomous AI Applications. . . . 97

Summary. . . . . . . . . . . . . . 98

Test Your Skills. . . . . . . . . . . . . 99

Exercise 3-1: Hugging Face.. . . . . . . . . . . 102

Exercise 3-2: Transformers in AI. . . . . . . . . . 102

Additional Resources. . . . . . . . . . . . 104

4 The Cornerstones of AI and ML Security.. . . . . . . . 105

Recognizing the Need for AI Security.. . . . . . . . . 105

Adversarial Attacks.. . . . . . . . . . . . 107

Data Poisoning Attacks. . . . . . . . . . . 109

OWASP Top Ten for LLMs. . . . . . . . . . . 112

Countermeasures Against Model Stealing Attacks. . . . . . 124

Membership Inference Attacks.. . . . . . . . . . 124

Evasion Attacks.. . . . . . . . . . . . . 130

Model Inversion Attacks.. . . . . . . . . . . 135

Backdoor Attacks. . . . . . . . . . . . . 137

Exploring Defensive Measures.. . . . . . . . . . 137

Summary. . . . . . . . . . . . . . 138

Test Your Skills. . . . . . . . . . . . . 140

Additional Resources. . . . . . . . . . . . 142

5 Hacking AI Systems.. . . . . . . . . . . . 143

Hacking FakeMedAI. . . . . . . . . . . . 143

MITRE ATLAS.. . . . . . . . . . . . . 146

A Deep Dive into the AI and ML Attack Tactics and Techniques.. . . . 147

Exploiting Prompt Injection.. . . . . . . . . . 161

Red-Teaming AI Models. . . . . . . . . . . 162

Summary. . . . . . . . . . . . . . 162

Test Your Skills. . . . . . . . . . . . . 163

Exercise 5-1: Understanding the MITRE ATT&CK Framework.. . . . 166

Exercise 5-2: Exploring the MITRE ATLAS Framework.. . . . . . 166

6 System and Infrastructure Security. . . . . . . . . 167

The Vulnerabilities and Risks Associated with AI Systems and Their Potential Impact. . . . . . . . . . . . 167

AI BOMs. . . . . . . . . . . . . . 176

Data Security Vulnerabilities.. . . . . . . . . . 177

Cloud Security Vulnerabilities.. . . . . . . . . . 179

Secure Design Principles for AI Systems.. . . . . . . . 182

AI Model Security.. . . . . . . . . . . . 183

Infrastructure Security for AI Systems. . . . . . . . . 184

Threat Detection and Incident Response for AI Systems.. . . . . 190

Additional Security Technologies and Considerations for AI Systems. . . 193

Summary. . . . . . . . . . . . . . 195

Test Your Skills. . . . . . . . . . . . . 196

Additional Resources. . . . . . . . . . . . 197

7 Privacy and Ethics: Navigating Privacy and Ethics in an AI-Infused World.. . . . . . . . . . . . . 199

Why Do We Need to Balance the Benefits of AI with the Ethical Risks and Privacy Concerns?.. . . . . . . . . . 200

What Are the Challenges Posed by AI in Terms of Privacy Protection, and What Is the Importance of Privacy and Ethics in AI Development and Deployment?. . . . . . . . . 201

The Dark Side of AI and ChatGPT: Privacy Concerns and Ethical Implications. . . . . . . . . . . . 203

Data Collection and Data Storage in AI Algorithms: Potential Risks and Ethical Privacy Concerns. . . . . . . . 206

The Moral Tapestry of AI and ChatGPT.. . . . . . . . 212

Preserving Privacy, Unleashing Knowledge: Differential Privacy and Federated Learning in the Age of Data Security.. . . . . . 223

Harmony in the Machine: Nurturing Fairness, Diversity, and Human Control in AI Systems.. . . . . . . . . . 224

Real-World Case Study Examples and Fictional Stories of Privacy Breaches in AI and ChatGPT.. . . . . . . . . 226

Summary. . . . . . . . . . . . . . 232

Test Your Skills. . . . . . . . . . . . . 234

Exercise 7-1: Privacy Concerns and Ethical Implications of AI. . . . 237

Exercise 7-2: Ethical Privacy Concerns in Data Collection and Storage by AI Algorithms.. . . . . . . . . . . . . 238

Exercise 7-3: Balancing Autonomy and Privacy in the Age of AI. . . . 238

Exercise 7-4: Safeguarding Privacy and Ethical Frontiers.. . . . . 239

8 Legal and Regulatory Compliance for AI Systems.. . . . . 241

Legal and Regulatory Landscape.. . . . . . . . . 242

Compliance with AI Legal and Regulatory Data Protection Laws. . . 245

Intellectual Property Issues in Conversational AI.. . . . . . 247

Unraveling Liability and Accountability in the Age of AI. . . . . 249

Ethical Development and Deployment of AI Systems: Strategies for Effective Governance and Risk Management.. . . . . . 252

International Collaboration and Standards in AI. . . . . . . 255

Future Trends and Outlook in AI Compliance.. . . . . . . 257

Unleashing the Quantum Storm: Fictional Story on AI Cybersecurity, Quantum Computing, and Novel Cyberattacks in Oxford, 2050. . . . 260

Summary. . . . . . . . . . . . . . 261

Test Your Skills. . . . . . . . . . . . . 263

Exercise 8-1: Compliance with Legal and Regulatory Data Protection Laws.. . . . . . . . . . . . 266

Exercise 8-2: Understanding Liability and Accountability in AI Systems. . . . . . . . . . . . . . 266

Exercise 8-3: International Collaboration and Standards in AI.. . . . 267

Test Your Skills Answers and Solutions. . . . . . . . . 269

 

9780138268459, TOC, 1/4/2024