Security Engineering

ROSS ANDERSON is Professor of Security Engineering at Cambridge University in England. He is widely recognized as one of the world's foremost authorities on security. In 2015 he won the Lovelace Medal, Britain's top award in computing. He is a Fellow of the Royal Society and the Royal Academy of Engineering. He is one of the pioneers of the economics of information security, peer-to-peer systems, API analysis and hardware security. Over the past 40 years, he has also worked or consulted for most of the tech majors.

Preface to the Third Edition xxxvii
Preface to the Second Edition xli
Preface to the First Edition xliii
Formy daughter, and other lawyers... xlvii
Foreword xlix

Part I
Chapter 1 What Is Security Engineering? 3
1.1 Introduction 3
1.2 A framework 4
1.3 Example 1 - a bank 6
1.4 Example 2 - a military base 7
1.5 Example 3 - a hospital 8
1.6 Example 4 - the home 10
1.7 Definitions 11
1.8 Summary 16

Chapter 2 Who Is the Opponent? 17
2.1 Introduction 17
2.2 Spies 19
2.2.1 The Five Eyes 19
2.2.1.1 Prism 19
2.2.1.2 Tempora 20
2.2.1.3 Muscular 21
2.2.1.4 Special collection 22
2.2.1.5 Bullrun and Edgehill 22
2.2.1.6 Xkeyscore 23
2.2.1.7 Longhaul 24
2.2.1.8 Quantum 25
2.2.1.9 CNE 25
2.2.1.10 The analyst's viewpoint 27
2.2.1.11 Offensive operations 28
2.2.1.12 Attack scaling 29
2.2.2 China 30
2.2.3 Russia 35
2.2.4 The rest 38
2.2.5 Attribution 40
2.3 Crooks 41
2.3.1 Criminal infrastructure 42
2.3.1.1 Botnet herders 42
2.3.1.2 Malware devs 44
2.3.1.3 Spam senders 45
2.3.1.4 Bulk account compromise 45
2.3.1.5 Targeted attackers 46
2.3.1.6 Cashout gangs 46
2.3.1.7 Ransomware 47
2.3.2 Attacks on banking and payment systems 47
2.3.3 Sectoral cybercrime ecosystems 49
2.3.4 Internal attacks 49
2.3.5 CEO crimes 49
2.3.6 Whistleblowers 50
2.4 Geeks 52
2.5 The swamp 53
2.5.1 Hacktivism and hate campaigns 54
2.5.2 Child sex abuse material 55
2.5.3 School and workplace bullying 57
2.5.4 Intimate relationship abuse 57
2.6 Summary 59
Research problems 60
Further reading 61

Chapter 3 Psychology and Usability 63
3.1 Introduction 63
3.2 Insights from psychology research 64
3.2.1 Cognitive psychology 65
3.2.2 Gender, diversity and interpersonal variation 68
3.2.3 Social psychology 70
3.2.3.1 Authority and its abuse 71
3.2.3.2 The bystander effect 72
3.2.4 The social-brain theory of deception 73
3.2.5 Heuristics, biases and behavioural economics 76
3.2.5.1 Prospect theory and risk misperception 77
3.2.5.2 Present bias and hyperbolic discounting 78
3.2.5.3 Defaults and nudges 79
3.2.5.4 The default to intentionality 79
3.2.5.5 The affect heuristic 80
3.2.5.6 Cognitive dissonance 81
3.2.5.7 The risk thermostat 81
3.3 Deception in practice 81
3.3.1 The salesman and the scamster 82
3.3.2 Social engineering 84
3.3.3 Phishing 86
3.3.4 Opsec 88
3.3.5 Deception research 89
3.4 Passwords 90
3.4.1 Password recovery 92
3.4.2 Password choice 94
3.4.3 Difficulties with reliable password entry 94
3.4.4 Difficulties with remembering the password 95
3.4.4.1 Naive choice 96
3.4.4.2 User abilities and training 96
3.4.4.3 Design errors 98
3.4.4.4 Operational failures 100
3.4.4.5 Social-engineering attacks 101
3.4.4.6 Customer education 102
3.4.4.7 Phishing warnings 103
3.4.5 System issues 104
3.4.6 Can you deny service? 105
3.4.7 Protecting oneself or others? 105
3.4.8 Attacks on password entry 106
3.4.8.1 Interface design 106
3.4.8.2 Trusted path, and bogus terminals 107
3.4.8.3 Technical defeats of password retry counters 107
3.4.9 Attacks on password storage 108
3.4.9.1 One-way encryption 109
3.4.9.2 Password cracking 109
3.4.9.3 Remote password checking 109
3.4.10 Absolute limits 110
3.4.11 Using a password manager 111
3.4.12 Will we ever get rid of passwords? 113
3.5 CAPTCHAs 115
3.6 Summary 116
Research problems 117
Further reading 118

Chapter 4 Protocols 119
4.1 Introduction 119
4.2 Password eavesdropping risks 120
4.3 Who goes there? - simple authentication 122
4.3.1 Challenge and response 124
4.3.2 Two-factor authentication 128
4.3.3 The MIG-in-the-middle attack 129
4.3.4 Reflection attacks 132
4.4 Manipulating the message 133
4.5 Changing the environment 134
4.6 Chosen protocol attacks 135
4.7 Managing encryption keys 136
4.7.1 The resurrecting duckling 137
4.7.2 Remote key management 137
4.7.3 The Needham-Schroeder protocol 138
4.7.4 Kerberos 139
4.7.5 Practical key management 141
4.8 Design assurance 141
4.9 Summary 143
Research problems 143
Further reading 144
Chapter 5 Cryptography 145
5.1 Introduction 145
5.2 Historical background 146
5.2.1 An early stream cipher - the Vigenere 147
5.2.2 The one-time pad 148
5.2.3 An early block cipher - Playfair 150
5.2.4 Hash functions 152
5.2.5 Asymmetric primitives 154
5.3 Security models 155
5.3.1 Random functions - hash functions 157
5.3.1.1 Properties 157
5.3.1.2 The birthday theorem 158
5.3.2 Random generators - stream ciphers 159
5.3.3 Random permutations - block ciphers 161
5.3.4 Public key encryption and trapdoor one-way permutations 163
5.3.5 Digital signatures 164
5.4 Symmetric crypto algorithms 165
5.4.1 SP-networks 165
5.4.1.1 Block size 166
5.4.1.2 Number of rounds 166
5.4.1.3 Choice of S-boxes 167
5.4.1.4 Linear cryptanalysis 167
5.4.1.5 Differential cryptanalysis 168
5.4.2 The Advanced Encryption Standard (AES) 169
5.4.3 Feistel ciphers 171
5.4.3.1 The Luby-Rackoff result 173
5.4.3.2 DES 173
5.5 Modes of operation 175
5.5.1 How not to use a block cipher 176
5.5.2 Cipher block chaining 177
5.5.3 Counter encryption 178
5.5.4 Legacy stream cipher modes 178
5.5.5 Message authentication code 179
5.5.6 Galois counter mode 180
5.5.7 XTS 180
5.6 Hash functions 181
5.6.1 Common hash functions 181
5.6.2 Hash function applications - HMAC, commitments and updating 183
5.7 Asymmetric crypto primitives 185
5.7.1 Cryptography based on factoring 185
5.7.2 Cryptography based on discrete logarithms 188
5.7.2.1 One-way commutative encryption 189
5.7.2.2 Diffie-Hellman key establishment 190
5.7.2.3 ElGamal digital signature and DSA 192
5.7.3 Elliptic curve cryptography 193
5.7.4 Certification authorities 194
5.7.5 TLS 195
5.7.5.1 TLS uses 196
5.7.5.2 TLS security 196
5.7.5.3 TLS 1.3 197
5.7.6 Other public-key protocols 197
5.7.6.1 Code signing 197
5.7.6.2 PGP/GPG 198
5.7.6.3 QUIC 199
5.7.7 Special-purpose primitives 199
5.7.8 How strong are asymmetric cryptographic primitives? 200
5.7.9 What else goes wrong 202
5.8 Summary 203
Research problems 204
Further reading 204

Chapter 6 Access Control 207
6.1 Introduction 207
6.2 Operating system access controls 209
6.2.1 Groups and roles 210
6.2.2 Access control lists 211
6.2.3 Unix operating system security 212
6.2.4 Capabilities 214
6.2.5 DAC and MAC 215
6.2.6 Apple's macOS 217
6.2.7 iOS 217
6.2.8 Android 218
6.2.9 Windows 219
6.2.10 Middleware 222
6.2.10.1 Database access controls 222
6.2.10.2 Browsers 223
6.2.11 Sandboxing 224
6.2.12 Virtualisation 225
6.3 Hardware protection 227
6.3.1 Intel processors 228
6.3.2 Arm processors 230
6.4 What goes wrong 231
6.4.1 Smashing the stack 232
6.4.2 Other technical attacks 234
6.4.3 User interface failures 236
6.4.4 Remedies 237
6.4.5 Environmental creep 238
6.5 Summary 239
Research problems 240
Further reading 240

Chapter 7 Distributed Systems 243
7.1 Introduction 243
7.2 Concurrency 244
7.2.1 Using old data versus paying to propagate state 245
7.2.2 Locking to prevent inconsistent updates 246
7.2.3 The order of updates 247
7.2.4 Deadlock 248
7.2.5 Non-convergent state 249
7.2.6 Secure time 250
7.3 Fault tolerance and failure recovery 251
7.3.1 Failure models 252
7.3.1.1 Byzantine failure 252
7.3.1.2 Interaction with fault tolerance 253
7.3.2 What is resilience for? 254
7.3.3 At what level is the redundancy? 255
7.3.4 Service-denial attacks 257
7.4 Naming 259
7.4.1 The Needham naming principles 260
7.4.2 What else goes wrong 263
7.4.2.1 Naming and identity 264
7.4.2.2 Cultural assumptions 265
7.4.2.3 Semantic content of names 267
7.4.2.4 Uniqueness of names 268
7.4.2.5 Stability of names and addresses 269
7.4.2.6 Restrictions on the use of names 269
7.4.3 Types of name 270
7.5 Summary 271
Research problems 272
Further reading 273

Chapter 8 Economics 275
8.1 Introduction 275
8.2 Classical economics 276
8.2.1 Monopoly 278
8.3 Information economics 281
8.3.1 Why information markets are different 281
8.3.2 The value of lock-in 282
8.3.3 Asymmetric information 284
8.3.4 Public goods 285
8.4 Game theory 286
8.4.1 The prisoners' dilemma 287
8.4.2 Repeated and evolutionary games 288
8.5 Auction theory 291
8.6 The economics of security and dependability 293
8.6.1 Why is Windows so insecure? 294
8.6.2 Managing the patching cycle 296
8.6.3 Structural models of attack and defence 298
8.6.4 The economics of lock-in, tying and DRM 300
8.6.5 Antitrust law and competition policy 302
8.6.6 Perversely motivated guards 304
8.6.7 Economics of privacy 305
8.6.8 Organisations and human behaviour 307
8.6.9 Economics of cybercrime 308
8.7 Summary 310
Research problems 311
Further reading 311

Part II
Chapter 9 Multilevel Security 315
9.1 Introduction 315
9.2 What is a security policy model? 316
9.3 Multilevel security policy 318
9.3.1 The Anderson report 319
9.3.2 The Bell-LaPadula model 320
9.3.3 The standard criticisms of Bell-LaPadula 321
9.3.4 The evolution of MLS policies 323
9.3.5 The Biba model 325
9.4 Historical examples of MLS systems 326
9.4.1 SCOMP 326
9.4.2 Data diodes 327
9.5 MAC: from MLS to IFC and integrity 329
9.5.1 Windows 329
9.5.2 SELinux 330
9.5.3 Embedded systems 330
9.6 What goes wrong 331
9.6.1 Composability 331
9.6.2 The cascade problem 332
9.6.3 Covert channels 333
9.6.4 The threat from malware 333
9.6.5 Polyinstantiation 334
9.6.6 Practical problems with MLS 335
9.7 Summary 337
Research problems 338
Further reading 339

Chapter 10 Boundaries 341
10.1 Introduction 341
10.2 Compartmentation and the lattice model 344
10.3 Privacy for tigers 346
10.4 Health record privacy 349
10.4.1 The threat model 351
10.4.2 The BMA security policy 353
10.4.3 First practical steps 356
10.4.4 What actually goes wrong 357
10.4.4.1 Emergency care 358
10.4.4.2 Resilience 359
10.4.4.3 Secondary uses 359
10.4.5 Confidentiality - the future 362
10.4.6 Ethics 365
10.4.7 Social care and education 367
10.4.8 The Chinese Wall 369
10.5 Summary 371
Research problems 372
Further reading 373

Chapter 11 Inference Control 375
11.1 Introduction 375
11.2 The early history of inference control 377
11.2.1 The basic theory of inference control 378
11.2.1.1 Query set size control 378
11.2.1.2 Trackers 379
11.2.1.3 Cell suppression 379
11.2.1.4 Other statistical disclosure control mechanisms 380
11.2.1.5 More sophisticated query controls 381
11.2.1.6 Randomization 382
11.2.2 Limits of classical statistical security 383
11.2.3 Active attacks 384
11.2.4 Inference control in rich medical data 385
11.2.5 The third wave: preferences and search 388
11.2.6 The fourth wave: location and social 389
11.3 Differential privacy 392
11.4 Mind the gap? 394
11.4.1 Tactical anonymity and its problems 395
11.4.2 Incentives 398
11.4.3 Alternatives 399
11.4.4 The dark side 400
11.5 Summary 401
Research problems 402
Further reading 402

Chapter 12 Banking and Bookkeeping 405
12.1 Introduction 405
12.2 Bookkeeping systems 406
12.2.1 Double-entry bookkeeping 408
12.2.2 Bookkeeping in banks 408
12.2.3 The Clark-Wilson security policy model 410
12.2.4 Designing internal controls 411
12.2.5 Insider frauds 415
12.2.6 Executive frauds 416
12.2.6.1 The post office case 418
12.2.6.2 Other failures 419
12.2.6.3 Ecological validity 420
12.2.6.4 Control tuning and corporate governance 421
12.2.7 Finding the weak spots 422
12.3 Interbank payment systems 424
12.3.1 A telegraphic history of E-commerce 424
12.3.2 SWIFT 425
12.3.3 What goes wrong 427
12.4 Automatic teller machines 430
12.4.1 ATM basics 430
12.4.2 What goes wrong 433
12.4.3 Incentives and injustices 437
12.5 Credit cards 438
12.5.1 Credit card fraud 439
12.5.2 Online card fraud 440
12.5.3 3DS 443
12.5.4 Fraud engines 444
12.6 EMV payment cards 445
12.6.1 Chip cards 445
12.6.1.1 Static data authentication 446
12.6.1.2 ICVVs, DDA and CDA 450
12.6.1.3 The No-PIN attack 451
12.6.2 The preplay attack 452
12.6.3 Contactless 454
12.7 Online banking 457
12.7.1 Phishing 457
12.7.2 CAP 458
12.7.3 Banking malware 459
12.7.4 Phones as second factors 459
12.7.5 Liability 461
12.7.6 Authorised push payment fraud 462
12.8 Nonbank payments 463
12.8.1 M-Pesa 463
12.8.2 Other phone payment systems 464
12.8.3 Sofort, and open banking 465
12.9 Summary 466
Research problems 466
Further reading 468

Chapter 13 Locks and Alarms 471
13.1 Introduction 471
13.2 Threats and barriers 472
13.2.1 Threat model 473
13.2.2 Deterrence 474
13.2.3 Walls and barriers 476
13.2.4 Mechanical locks 478
13.2.5 Electronic locks 482
13.3 Alarms 484
13.3.1 How not to protect a painting 485
13.3.2 Sensor defeats 486
13.3.3 Feature interactions 488
13.3.4 Attacks on communications 489
13.3.5 Lessons learned 493
13.4 Summary 494
Research problems 495
Further reading 495

Chapter 14 Monitoring and Metering 497
14.1 Introduction 497
14.2 Prepayment tokens 498
14.2.1 Utility metering 499
14.2.2 How the STS system works 501
14.2.3 What goes wrong 502
14.2.4 Smart meters and smart grids 504
14.2.5 Ticketing fraud 508
14.3 Taxi meters, tachographs and truck speed limiters 509
14.3.1 The tachograph 509
14.3.2 What goes wrong 511
14.3.2.1 How most tachograph manipulation is done 511
14.3.2.2 Tampering with the supply 512
14.3.2.3 Tampering with the instrument 512
14.3.2.4 High-tech attacks 513
14.3.3 Digital tachographs 514
14.3.3.1 System-level problems 515
14.3.3.2 Other problems 516
14.3.4 Sensor defeats and third-generation devices 518
14.3.5 The fourth generation - smart tachographs 518
14.4 Curfew tags: GPS as policeman 519
14.5 Postage meters 522
14.6 Summary 526
Research problems 527
Further reading 527

Chapter 15 Nuclear Command and Control 529
15.1 Introduction 529
15.2 The evolution of command and control 532
15.2.1 The Kennedy memorandum 532
15.2.2 Authorization, environment, intent 534
15.3 Unconditionally secure authentication 534
15.4 Shared control schemes 536
15.5 Tamper resistance and PALs 538
15.6 Treaty verification 540
15.7 What goes wrong 541
15.7.1 Nuclear accidents 541
15.7.2 Interaction with cyberwar 542
15.7.3 Technical failures 543
15.8 Secrecy or openness? 544
15.9 Summary 545
Research problems 546
Further reading 546

Chapter 16 Security Printing and Seals 549
16.1 Introduction 549
16.2 History 550
16.3 Security printing 551
16.3.1 Threat model 552
16.3.2 Security printing techniques 553
16.4 Packaging and seals 557
16.4.1 Substrate properties 558
16.4.2 The problems of glue 558
16.4.3 PIN mailers 559
16.5 Systemic vulnerabilities 560
16.5.1 Peculiarities of the threat model 562
16.5.2 Anti-gundecking measures 563
16.5.3 The effect of random failure 564
16.5.4 Materials control 564
16.5.5 Not protecting the right things 565
16.5.6 The cost and nature of inspection 566
16.6 Evaluation methodology 567
16.7 Summary 569
Research problems 569
Further reading 570

Chapter 17 Biometrics 571
17.1 Introduction 571
17.2 Handwritten signatures 572
17.3 Face recognition 575
17.4 Fingerprints 579
17.4.1 Verifying positive or negative identity claims 581
17.4.2 Crime scene forensics 584
17.5 Iris codes 588
17.6 Voice recognition and morphing 590
17.7 Other systems 591
17.8 What goes wrong 593
17.9 Summary 596
Research problems 597
Further reading 597

Chapter 18 Tamper Resistance 599
18.1 Introduction 599
18.2 History 601
18.3 Hardware security modules 601
18.4 Evaluation 607
18.5 Smartcards and other security chips 609
18.5.1 History 609
18.5.2 Architecture 610
18.5.3 Security evolution 611
18.5.4 Random number generators and PUFs 621
18.5.5 Larger chips 624
18.5.6 The state of the art 628
18.6 The residual risk 630
18.6.1 The trusted interface problem 630
18.6.2 Conflicts 631
18.6.3 The lemons market, risk dumping and evaluation games 632
18.6.4 Security-by-obscurity 632
18.6.5 Changing environments 633
18.7 So what should one protect? 634
18.8 Summary 636
Research problems 636
Further reading 636

Chapter 19 Side Channels 639
19.1 Introduction 639
19.2 Emission security 640
19.2.1 History 641
19.2.2 Technical surveillance and countermeasures 642
19.3 Passive attacks 645
19.3.1 Leakage through power and signal cables 645
19.3.2 Leakage through RF signals 645
19.3.3 What goes wrong 649
19.4 Attacks between and within computers 650
19.4.1 Timing analysis 651
19.4.2 Power analysis 652
19.4.3 Glitching and differential fault analysis 655
19.4.4 Rowhammer, CLKscrew and Plundervolt 656
19.4.5 Meltdown, Spectre and other enclave side channels 657
19.5 Environmental side channels 659
19.5.1 Acoustic side channels 659
19.5.2 Optical side channels 661
19.5.3 Other side-channels 661
19.6 Social side channels 663
19.7 Summary 663
Research problems 664
Further reading 664

Chapter 20 Advanced Cryptographic Engineering 667
20.1 Introduction 667
20.2 Full-disk encryption 668
20.3 Signal 670
20.4 Tor 674
20.5 HSMs 677
20.5.1 The xor-to-null-key attack 677
20.5.2 Attacks using backwards compatibility and time-memory tradeoffs 678
20.5.3 Differential protocol attacks 679
20.5.4 The EMV attack 681
20.5.5 Hacking the HSMs in CAs and clouds 681
20.5.6 Managing HSM risks 681
20.6 Enclaves 682
20.7 Blockchains 685
20.7.1 Wallets 688
20.7.2 Miners 689
20.7.3 Smart contracts 689
20.7.4 Off-chain payment mechanisms 691
20.7.5 Exchanges, cryptocrime and regulation 692
20.7.6 Permissioned blockchains 695
20.8 Crypto dreams that failed 695
20.9 Summary 696
Research problems 698
Further reading 698

Chapter 21 Network Attack and Defence 699
21.1 Introduction 699
21.2 Network protocols and service denial 701
21.2.1 BGP security 701
21.2.2 DNS security 703
21.2.3 UDP, TCP, SYN floods and SYN reflection 704
21.2.4 Other amplifiers 705
21.2.5 Other denial-of-service attacks 706
21.2.6 Email - from spies to spammers 706
21.3 The malware menagerie - Trojans, worms and RATs 708
21.3.1 Early history of malware 709
21.3.2 The Internet worm 710
21.3.3 Further malware evolution 711
21.3.4 How malware works 713
21.3.5 Countermeasures 714
21.4 Defense against network attack 715
21.4.1 Filtering: firewalls, censorware and wiretaps 717
21.4.1.1 Packet filtering 718
21.4.1.2 Circuit gateways 718
21.4.1.3 Application proxies 719
21.4.1.4 Ingress versus egress filtering 719
21.4.1.5 Architecture 720
21.4.2 Intrusion detection 722
21.4.2.1 Types of intrusion detection 722
21.4.2.2 General limitations of intrusion detection 724
21.4.2.3 Specific problems detecting network attacks 724
21.5 Cryptography: the ragged boundary 725
21.5.1 SSH 726
21.5.2 Wireless networking at the periphery 727
21.5.2.1 WiFi 727
21.5.2.2 Bluetooth 728
21.5.2.3 HomePlug 729
21.5.2.4 VPNs 729
21.6 CAs and PKI 730
21.7 Topology 733
21.8 Summary 734
Research problems 734
Further reading 735

Chapter 22 Phones 737
22.1 Introduction 737
22.2 Attacks on phone networks 738
22.2.1 Attacks on phone-call metering 739
22.2.2 Attacks on signaling 742
22.2.3 Attacks on switching and configuration 743
22.2.4 Insecure end systems 745
22.2.5 Feature interaction 746
22.2.6 VOIP 747
22.2.7 Frauds by phone companies 748
22.2.8 Security economics of telecomms 749
22.3 Going mobile 750
22.3.1 GSM 751
22.3.2 3G 755
22.3.3 4G 757
22.3.4 5G and beyond 758
22.3.5 General MNO failings 760
22.4 Platform security 761
22.4.1 The Android app ecosystem 763
22.4.1.1 App markets and developers 764
22.4.1.2 Bad Android implementations 764
22.4.1.3 Permissions 766
22.4.1.4 Android malware 767
22.4.1.5 Ads and third-party services 768
22.4.1.6 Pre-installed apps 770
22.4.2 Apple's app ecosystem 770
22.4.3 Cross-cutting issues 774
22.5 Summary 775
Research problems 776
Further reading 776

Chapter 23 Electronic and Information Warfare 777
23.1 Introduction 777
23.2 Basics 778
23.3 Communications systems 779
23.3.1 Signals intelligence techniques 781
23.3.2 Attacks on communications 784
23.3.3 Protection techniques 785
23.3.3.1 Frequency hopping 786
23.3.3.2 DSSS 787
23.3.3.3 Burst communications 788
23.3.3.4 Combining covertness and jam resistance 789
23.3.4 Interaction between civil and military uses 790
23.4 Surveillance and target acquisition 791
23.4.1 Types of radar 792
23.4.2 Jamming techniques 793
23.4.3 Advanced radars and countermeasures 795
23.4.4 Other sensors and multisensor issues 796
23.5 IFF systems 797
23.6 Improvised explosive devices 800
23.7 Directed energy weapons 802
23.8 Information warfare 803
23.8.1 Attacks on control systems 805
23.8.2 Attacks on other infrastructure 808
23.8.3 Attacks on elections and political stability 809
23.8.4 Doctrine 811
23.9 Summary 812
Research problems 813
Further reading 813

Chapter 24 Copyright and DRM 815
24.1 Introduction 815
24.2 Copyright 817
24.2.1 Software 817
24.2.2 Free software, free culture? 823
24.2.3 Books and music 827
24.2.4 Video and pay-TV 828
24.2.4.1 Typical system architecture 829
24.2.4.2 Video scrambling techniques 830
24.2.4.3 Attacks on hybrid scrambling systems 832
24.2.4.4 DVB 836
24.2.5 DVD 837
24.3 DRM on general-purpose computers 838
24.3.1 Windows media rights management 839
24.3.2 FairPlay, HTML5 and other DRM systems 840
24.3.3 Software obfuscation 841
24.3.4 Gaming, cheating, and DRM 843
24.3.5 Peer-to-peer systems 845
24.3.6 Managing hardware design rights 847
24.4 Information hiding 848
24.4.1 Watermarks and copy generation management 849
24.4.2 General information hiding techniques 849
24.4.3 Attacks on copyright marking schemes 851
24.5 Policy 854
24.5.1 The IP lobby 857
24.5.2 Who benefits? 859
24.6 Accessory control 860
24.7 Summary 862
Research problems 862
Further reading 863

Chapter 25 New Directions? 865
25.1 Introduction 865
25.2 Autonomous and remotely-piloted vehicles 866
25.2.1 Drones 866
25.2.2 Self-driving cars 867
25.2.3 The levels and limits of automation 869
25.2.4 How to hack a self-driving car 872
25.3 AI / ML 874
25.3.1 ML and security 875
25.3.2 Attacks on ML systems 876
25.3.3 ML and society 879
25.4 PETS and operational security 882
25.4.1 Anonymous messaging devices 885
25.4.2 Social support 887
25.4.3 Living off the land 890
25.4.4 Putting it all together 891
25.4.5 The name's Bond. James Bond 893
25.5 Elections 895
25.5.1 The history of voting machines 896
25.5.2 Hanging chads 896
25.5.3 Optical scan 898
25.5.4 Software independence 899
25.5.5 Why electronic elections are hard 900
25.6 Summary 904
Research problems 904
Further reading 905

Part III
Chapter 26 Surveillance or Privacy? 909
26.1 Introduction 909
26.2 Surveillance 912
26.2.1 The history of government wiretapping 912
26.2.2 Call data records (CDRs) 916
26.2.3 Search terms and location data 919
26.2.4 Algorithmic processing 920
26.2.5 ISPs and CSPs 921
26.2.6 The Five Eyes' system of systems 922
26.2.7 The crypto wars 925
26.2.7.1 The back story to crypto policy 926
26.2.7.2 DES and crypto research 927
26.2.7.3 CryptoWar 1 - the Clipper chip 928
26.2.7.4 CryptoWar 2 - going spotty 931
26.2.8 Export control 934
26.3 Terrorism 936
26.3.1 Causes of political violence 936
26.3.2 The psychology of political violence 937
26.3.3 The role of institutions 938
26.3.4 The democratic response 940
26.4 Censorship 941
26.4.1 Censorship by authoritarian regimes 942
26.4.2 Filtering, hate speech and radicalisation 944
26.5 Forensics and rules of evidence 948
26.5.1 Forensics 948
26.5.2 Admissibility of evidence 950
26.5.3 What goes wrong 951
26.6 Privacy and data protection 953
26.6.1 European data protection 953
26.6.2 Privacy regulation in the USA 956
26.6.3 Fragmentation? 958
26.7 Freedom of information 960
26.8 Summary 961
Research problems 962
Further reading 962

Chapter 27 Secure Systems Development 965
27.1 Introduction 965
27.2 Risk management 966
27.3 Lessons from safety-critical systems 969
27.3.1 Safety engineering methodologies 970
27.3.2 Hazard analysis 971
27.3.3 Fault trees and threat trees 971
27.3.4 Failure modes and effects analysis 972
27.3.5 Threat modelling 973
27.3.6 Quantifying risks 975
27.4 Prioritising protection goals 978
27.5 Methodology 980
27.5.1 Top-down design 981
27.5.2 Iterative design: from spiral to agile 983
27.5.3 The secure development lifecycle 985
27.5.4 Gated development 987
27.5.5 Software as a Service 988
27.5.6 From DevOps to DevSecOps 991
27.5.6.1 The Azure ecosystem 991
27.5.6.2 The Google ecosystem 992
27.5.6.3 Creating a learning system 994
27.5.7 The vulnerability cycle 995
27.5.7.1 The CVE system 997
27.5.7.2 Coordinated disclosure 998
27.5.7.3 Security incident and event management 999
27.5.8 Organizational mismanagement of risk 1000
27.6 Managing the team 1004
27.6.1 Elite engineers 1004
27.6.2 Diversity 1005
27.6.3 Nurturing skills and attitudes 1007
27.6.4 Emergent properties 1008
27.6.5 Evolving your workflow 1008
27.6.6 And finally... 1010
27.7 Summary 1010
Research problems 1011
Further reading 1012

Chapter 28 Assurance and Sustainability 1015
28.1 Introduction 1015
28.2 Evaluation 1018
28.2.1 Alarms and locks 1019
28.2.2 Safety evaluation regimes 1019
28.2.3 Medical device safety 1020
28.2.4 Aviation safety 1023
28.2.5 The Orange book 1025
28.2.6 FIPS 140 and HSMs 1026
28.2.7 The common criteria 1026
28.2.7.1 The gory details 1027
28.2.7.2 What goes wrong with the Common Criteria 1029
28.2.7.3 Collaborative protection profiles 1031
28.2.8 The 'Principle of Maximum Complacency' 1032
28.2.9 Next steps 1034
28.3 Metrics and dynamics of dependability 1036
28.3.1 Reliability growth models 1036
28.3.2 Hostile review 1039
28.3.3 Free and open-source software 1040
28.3.4 Process assurance 1042
28.4 The entanglement of safety and security 1044
28.4.1 The electronic safety and security of cars 1046
28.4.2 Modernising safety and security regulation 1049
28.4.3 The Cybersecurity Act 2019 1050
28.5 Sustainability 1051
28.5.1 The Sales of goods directive 1052
28.5.2 New research directions 1053
28.6 Summary 1056
Research problems 1057
Further reading 1058

Chapter 29 Beyond "Computer Says No" 1059
Bibliography 1061
Index 1143