Malware Detection (eBook)

Preface 6
Contents 8
Introduction 10
Part I Overview 13
1 Malware Evolution: A Snapshot of Threats and Countermeasures in 2005 13
1.1 Overview 15
1.2 Evolution of Threats 15
1.3 Evolution of Countermeasures 17
1.4 Summary 23
References 23
Part II Software Analysis and Assurance 29
2 Static Disassembly and Code Analysis 31
2.1 Introduction 31
2.2 Robust Disassembly of Obfuscated Binaries 32
2.3 Code Analysis 46
2.4 Conclusions 52
References 53
3 A Next- Generation Platform for Analyzing Executables* 55
3.1 Introduction 56
3.2 Advantages of Analyzing Executables 57
3.3 Analyzing Executables in the Absence of Source Code 60
3.4 Model-Checking Facilities 69
3.5 Related Work 71
References 71
4 Behavioral and Structural Properties of Malicious Code 75
4.1 Introduction 75
4.2 Behavioral Identification of Rootliits 76
4.3 Structural Identification of Worms 83
4.4 Conclusions 94
References 94
5 Detection and Prevention of SQL Injection Attacks 97
5.1 Introduction 97
5.2 SQL Injection Attacks Explained 99
5.3 Detection and Prevention of SQL Injection Attacks 106
5.4 Empirical Evaluation 112
5.5 Related Approaches 117
5.6 Conclusion 119
Acknowledgments 120
References 120
Part III Distributed Threat Detection and Defense 123
6 Very Fast Containment of Scanning Worms, Revisited* 125
6.1 Introduction 125
6.2 Worm Containment 127
6.3 Scan Suppression 129
6.4 Hardware Implementations 130
6.5 Approximate Scan Suppression 133
6.6 Cooperation 142
6.7 Attacking Worm Containment 145
6.8 Related Work 149
6.9 Future Work 150
6.10 Conclusions 151
6.11 Revisited 151
6.12 Acknowledgments 156
References 156
7 Sting: An End- to- End Self-Healing System for Defending against Internet Worms 159
7.1 Introduction 159
7.2 Worm Defense Design Space 161
7.3 Dynamic Taint Analysis for Automatic Detection of New Exploits 162
7.4 Automatic Generation of Input-based Filters 165
7.5 Automatic Generation of Vulnerability-Specific Execution Filters 171
7.6 Sting Self-healing Architecture and Experience 172
7.7 Evaluation 174
7.8 Related Work 177
7.9 Conclusion 178
References 179
8 An Inside Look at Botnets 183
8.1 Introduction 183
8.2 Related Work 186
8.3 Evaluation 186
8.4 Conclusions 200
Acknowledgements 201
References 201
9 Can Cooperative Intrusion Detectors Challenge the Base- Rate Fallacy? 205
9.1 Introduction 205
9.2 Overview 206
9.3 The Problem of Detector Combination 209
9.4 Possible Solutions to the Detector-Combination Problem 210
9.5 Recommendations to IDS Developers 214
9.6 Related Work 218
9.7 Future Vision 220
References 220
Part IV Stealthy and Targeted Threat Detection and Defense 223
10 Composite Hybrid Techniques For Defending Against Targeted Attacks 225
10.1 Introduction 225
10.2 Architecture 227
10.3 Limitations 233
10.4 Related Work 233
10.5 Conclusion 237
References 237
11 Towards Stealthy Malware Detection 243
Abstract 243
11.1 Introduction 244
11.2 Deceiving anti-virus software 246
11.3 N-gram experiments on files 249
11.4 Concluding Remarks 259
References 260
Part V Novel Techniques for Constructing Trustworthy Services 263
12 Pioneer: Verifying Code Integrity and Enforcing Untampered Code Execution on Legacy Systems* 265
12.1 Introduction 265
12.2 Problem Definition, Assumptions & Attacker Model
12.3 Pioneer Overview 269
12.4 Design of the Checksum Code 271
12.5 Checksum Code Implementation on the Netburst Microarchitecture 278
12.6 Applications 293
12.7 Related Work 297
12.8 Conclusions and Future Work 299
12.9 Acknowledgments 300
References 300
13 Principles of Secure Information Flow Analysis 303
13.1 Basic Principles 304
13.2 Typing Principles 306
13.3 Challenges 315
13.4 Conclusion 317
References 317
Index 321

1.3.1 Countermeasures for Previously Unseen Threats (p. 6)
Countermeasures for previously unseen threats are addressed below first for detecting previously unseen threats against already known vulnerabilities and identifying previously unknown vulnerabilities, and then for detecting previously unseen threats without foreknowledge of the vulnerability.

Blocking Previously Unseen Threats Against Already Known Vulnerabilities
Techniques such as Generic Exploit Blocking (GEB) [33] and Microsoft's Shield effort [47] were conceived to provide protection against previously unseen threats. These techniques use analysis of a known vulnerability to produce a signature that is not specific to any single instance of malware exploiting the vulnerability.

Thus, such a properly written signature can properly detect all potential attacks against a given vulnerability. This is in contrast with traditional antivirus and IDS heuristics which may be able to detect a percentage of new threats, but cannot guarantee complete detection. However, these approaches include a number of challenges in implementation, including the following three challenges.

- First, the signatures must be specified in a language and processed by a scanning engine that facilitate "performanf' scanning, either in the sense of high line-speeds, as is the constraint for traditional intrusion detection and network level anti-virus systems, or in the sense of low CPU burden.

- Second, the system must maintain low false positives while producing high true positives.

- Third, even though these approaches do not require prior knowledge of the malware, they still require prior knowledge of the vulnerability. The luxury of that prior knowledge is not always available. The next two sections describe techniques for identifying previously unknown vulnerabilities, and techniques for detecting previously unseen threats without the luxury of knowledge of the vulnerability.

Identifying Previously Unknown Vulnerabilities

Given that the above techniques rely on prior knowledge of vulnerabilities, they would be substantially more valuable if it was possible to better identify vulnerabilities in software before malware was created to exploit those vulnerabilities. A form of random test case generation known as Fuzzing [5] is among the most common techniques for finding vulnerabilities.

More recently, static analysis of the target software itself has been used to intelligently generate test cases more efficiently identifying vulnerabilities likely to exist near comer cases in target software execution [16, 23]. Although these techniques currently require source code, substantial progress has been made in extracting models from executable code for model checking and other static analysis without source code [13, 14]. However, in discussing static analysis of binaries, it is important to note that such tools can be used very effectively by creators of malware just as easily as they can be used by the security community [30].

Identifying Previously Unseen Threats without Prior Knowledge of Vulnerabilities

In this section we describe several emerging techniques that do not require prior knowledge of vulnerabilities for identifying previously unseen threats. These techniques include behavior based techniques, honeypots, anomaly detection, fault analysis, and correlation. Dynamic analysis of program behavior within a host is not new [11]. Behavior analysis was extended with various forms of anomaly detection [25] to improve generalization to previously unseen attacks while reducing false positives.