Blockchain Basics (eBook)

Daniel Drescher is an experienced banking professional who has held positions in electronic security trading in a range of banks. His recent activities have focused on automation, machine learning and big data in the context of security trading. Amongst others, Daniel holds a Doctorate in Econometrics from the Technical University of Berlin and an MSc in Software Engineering from the University of Oxford.

Contents 5
About the Author 7
About the Technical Reviewer 8
Introduction 9
Stage I:Terminology and Technical Foundations 14
Step1: Thinking in Layers and Aspects 15
The Metaphor 16
Layers of a Software System 16
Application vs. Implementation 16
Functional vs. Nonfunctional Aspects 17
Considering Two Layers at the Same Time 17
Integrity 18
Outlook 19
Summary 19
Step2: Seeing the Big Picture 20
The Metaphor 20
A Payment System 21
Two Types of Software Architecture 21
The Advantages of Distributed Systems 23
Higher Computing Power 23
Cost Reduction 23
Higher Reliability 23
Ability to Grow Naturally 24
The Disadvantages of Distributed Systems 24
Coordination Overhead 24
Communication Overhead 24
Dependencies on Networks 25
Higher Program Complexity 25
Security Issues 25
Distributed Peer-to-Peer Systems 25
Mixing Centralized and Distributed Systems 26
Identifying Distributed Systems 27
The Purpose of the Blockchain 27
Outlook 28
Summary 28
Step3: Recognizing the Potential 29
The Metaphor 29
How a Peer-to-Peer System Changed a Whole Industry 30
The Potential of Peer-to-Peer Systems 31
Terminology and the Link to the Blockchain 32
The Definition of a Peer-to-Peer System 33
Architecture of Peer-to-Peer Systems 33
The Link Between Peer-to-Peer Systems and the Blockchain 34
The Potential of the Blockchain 34
Outlook 34
Summary 35
Stage II:Why the Blockchain Is Needed 36
Step4: Discovering the Core Problem 37
The Metaphor 37
Trust and Integrity in Peer-to-Peer Systems 38
Integrity Threats in Peer-to-Peer Systems 38
Technical Failures 39
Malicious Peers 39
The Core Problem to Be Solved by the Blockchain 39
Outlook 40
Summary 40
Step5: Disambiguating the Term 41
The Term 41
A Data Structure 42
An Algorithm 42
A Suite of Technologies 42
An Umbrella Term for Purely Distributed Peer-to-Peer Systems with a Common Application Area 42
The Usage of the Term in This Book 43
Provisional Definition 43
The Role of Managing Ownership 43
The Application Area of the Blockchain in This Book 44
Outlook 44
Summary 45
Step6: Understanding the Nature of Ownership 46
The Metaphor 46
Ownership and Witnesses 47
Foundations of Ownership 48
A Short Detour to Security 49
Identification 50
Authentication 50
Authorization 50
Purposes and Properties of a Ledger 51
Ownership and the Blockchain 52
Outlook 53
Summary 53
Step7: Spending Money Twice 55
The Metaphor 55
The Double Spending Problem 56
The Term 57
Double Spending as a Problem of Copying Digital Goods 57
Double Spending as a Problem of Distributed Peer-to-Peer Systems of Ledgers 57
Double Spending as an Example of Violated Integrity in Distributed Peer-to-Peer Systems 58
How to Solve the Double Spending Problem 58
Solving Double Spending as a Problem of Copying Digital Goods 58
Solving Double Spending as a Problem of a Distributed Peer-to-Peer System of Ledgers 58
Solving Double Spending as an Example of Violated Integrity in Distributed Peer-to-Peer Systems 59
The Usage of Double Spending in This Book 59
Outlook 59
Summary 59
Stage III:How the Blockchain Works 61
Step8: Planning the Blockchain 62
The Goal 62
Starting Point 63
The Path to Follow 63
Task 1: Describing Ownership 64
Task 2: Protecting Ownership 64
Task 3: Storing Transaction Data 64
Task 4: Preparing Ledgers to Be Distributed in an Untrustworthy Environment 65
Task 5: Distributing the Ledgers 65
Task 6: Adding New Transactions to the Ledgers 65
Task 7: Deciding Which Ledgers Represent the Truth 66
Outlook 66
Summary 67
Step9: Documenting Ownership 68
The Metaphor 68
The Goal 69
The Challenge 69
The Idea 69
A Short Detour to Inventory and Transaction Data 69
How It Works 70
Describing the Transfer of Ownership 70
Maintaining the History of Transfers 71
Why It Works 71
Importance of Ordering 71
Integrity of the Transaction History 72
Formal Correctness 72
Semantic Correctness 72
Authorization 73
Outlook 73
Summary 73
Step10: Hashing Data 75
The Metaphor 75
The Goal 75
How It Works 76
Providing Hash Values for Any Data Quickly 76
Deterministic 76
Pseudorandom 77
One-Way Function 77
Collision Resistant 77
Trying It Out Yourself 77
Patterns of Hashing Data 79
Independent Hashing 79
Repeated Hashing 80
Combined Hashing 80
Sequential Hashing 81
Hierarchical Hashing 82
Outlook 83
Summary 83
Step11: Hashing in the Real World 84
Comparing Data 84
The Goal 84
The Idea 85
How It Works 85
Why It Works 85
Detecting Changes in Data 85
The Goal 85
The Idea 85
How It Works 85
Why It Works 86
Referring to Data in a Change-Sensitive Manner 86
The Goal 86
The Idea 86
How It Works 86
A Schematic Illustration 87
Why It Works 89
Storing Data in a Change-Sensitive Manner 89
The Goal 89
The Idea 89
How It Works 89
The Chain 90
The Tree 90
Why It Works 91
Causing Time-Consuming Computations 92
The Goal 92
The Idea 92
How It Works 92
An Illustrative Example 93
The Difficulty Level 94
Why It Works 94
Usage of Hashing in the Blockchain 95
Outlook 95
Summary 95
Step12: Identifying and Protecting User Accounts 96
The Metaphor 96
The Goal 97
The Challenge 97
The Idea 97
A Short Detour to Cryptography 98
The Major Idea of Cryptography 98
Terminology2 98
Symmetric Cryptography 99
Asymmetric Cryptography 99
Asymmetric Cryptography in the Real World 101
Creating and Distributing the Keys 101
Using the Keys 101
Public to Private 102
Private to Public 102
Asymmetric Cryptography in the Blockchain 102
Identifying Accounts 102
Authorizing Transactions 103
Outlook 103
Summary 103
Step13: Authorizing Transactions 105
The Metaphor 105
The Goal 106
The Challenge 106
The Idea 106
A Short Detour to Digital Signatures 106
Creating a Signature 107
Verifying Data by Using the Signature 107
Identifying Fraud by Using the Signature 108
How It Works 109
Signing a Transaction 109
Verifying a Transaction 110
Why It Works 110
Outlook 111
Summary 111
Step14: Storing Transaction Data 112
The Metaphor 113
The Goal 113
The Challenge 113
The Idea 113
Transforming a Book into a Blockchain-Data-Structure 114
Starting Point: A Book 114
Transformation 1: Making Page Dependency Explicit 114
Transformation 2: Outsourcing the Content 116
Transformation 3: Replacing Page Numbers 117
Transformation 4: Creating Reference Numbers 118
Transformation 5: Getting Rid of the Book Spine 118
Goal Achieved: Appreciating the Result 118
The Blockchain-Data-Structure 119
The Mental Unit of a Page of the Ordering Catalog and Its Corresponding Content Page 120
Ordering Catalog 120
Content Pages 121
Catalog Page Reference Numbers 121
Content Reference Numbers 121
Storing Transactions in the Blockchain-Data-Structure 121
Outlook 123
Summary 123
Step15: Using the Data Store 124
The Metaphor 124
Adding New Transactions 125
Detecting Changes 128
Changing the Content of Transaction Data 128
Changing a Reference in the Merkle Tree 129
Replacing a Transaction 129
Changing the Merkle Root 130
Changing a Block Header Reference 131
Changing Data Orderly 132
Intended vs. Unintended Changes 133
Outlook 134
Summary 134
Step 16: Protecting the Data Store 136
The Metaphor 136
The Goal 137
The Challenge 137
The Idea 138
A Short Detour to Immutability 138
How It Works: The Big Picture 138
Making Manipulations Stand Out 139
Enforcing Rewriting the History for Embedding Changes 139
Making Adding Data Computationally Expensive 139
How It Works: The Details 140
Compulsory Data 140
The Process of Creating A New Block 140
Validation Rules 141
Why It Works 142
The Costs of Manipulating the Blockchain-Data-Structure 142
The Immutable Data Store in the Real World 143
Outlook 144
Summary 144
Step 17:Distributing the Data Store Among Peers 145
The Metaphor 145
The Goal 146
The Challenge 146
The Idea 146
How It Works: The Overview1 147
How It Works: The Details 148
Keeping Existing Connections Alive 149
Establishing New Connections 149
Distributing New Information 149
Why It Works 150
Outlook 150
Summary 150
Step 18:Verif ying and Adding Transactions 153
The Metaphor 153
Consequences 154
The Goal 155
The Challenge 155
The Idea 155
How It Works: The Building Blocks 155
Validation Rules 156
Validation Rules for Transaction Data 156
Validation Rules for Block Headers 156
Reward 156
Punishment 157
Competition 157
Speed Competition 158
Quality Competition 158
Peer Control 158
How It Works: The Skeleton 159
How It Works: The Details 159
Why It Works 161
Dealing with Dishonest Behavior 162
Outlook 163
Summary 163
Step19: Choosing a Transaction History 165
The Metaphor 165
The Goal 166
The Challenge 166
The Idea 167
How It Works 168
The Longest-Chain-Criterion 168
The Heaviest-Chain-Criterion 173
Consequences of Selecting One Chain 174
Orphan Blocks 174
Reclaimed Reward 175
Clarifying Ownership 175
Reprocessing of Transactions 175
A Growing Common Trunk 176
Eventual Consistency 176
Robustness Against Manipulations 177
Threats to the Voting Schema 178
The Role of the Hash Puzzle 178
Why It Works 179
Outlook 179
Summary 179
Step20: Paying for Integrity 182
The Metaphor 182
The Role of Fees Within the Blockchain 183
Impact on the Integrity of the System 184
Impact on the Openness of the System 184
Impact on the Distributed Nature of the System 184
Impact on the Philosophy of the System 184
Desirable Properties of an Instrument of Payment for Compensating Peers 185
A Detour to the Emergence of Cryptographic Currencies 185
Outlook 186
Summary 186
Step21: Bringing the Pieces Together 188
Reviewing Concepts and Technologies 188
What Is the Blockchain? 190
The Purpose of the Blockchain: Functional Aspects of the Application Layer 191
Clarifying Ownership 191
Transferring Ownership 191
Properties of the Blockchain: Nonfunctional Aspects 192
Highly Available 192
Censorship Proof 192
Reliable 192
Open 192
Pseudoanonymous 192
Secure 193
Resilient 193
Eventually Consistent 193
Keeping Integrity 193
Internal Functioning: Functional Aspects of the Implementation Layer 193
Ownership Logic 194
Transaction Security 194
Transaction Processing Logic 195
Storage Logic 196
Peer-to-Peer Architecture 197
Consensus Logic 198
Gaining Abstraction 198
Outlook 199
Summary 199
Stage IV:Limitations and How to Overcome Them 202
Step22: Seeing the Limitations 203
The Challenge 203
Technical Limitations of the Blockchain 204
Lack of Privacy 204
The Security Model 204
Limited Scalability 205
High Costs 205
Hidden Centrality 206
Lack of Flexibility 206
Critical Size 207
Nontechnical Limitations of the Blockchain 207
Lack of Legal Acceptance 207
Lack of User Acceptance 208
Overcoming the Limitations 208
Technical Limitations 208
Nontechnical Limitations 208
Outlook 209
Summary 209
Step23: Reinventing the Blockchain 210
The Metaphor 210
Conflicting Goals of the Blockchain 211
Transparency vs. Privacy 211
Security vs. Speed 211
The Roots of the Conflicts 211
Solving the Conflicts 212
Deciding on Transparency vs. Privacy 212
Deciding on Security vs. Speed 213
Four Versions of the Blockchain 213
Consequences 214
The Peer-to-Peer Architecture 214
The Distributed Nature 214
Purpose 215
Reviewing the Purpose of the Blockchain 215
The Usage of the Term Blockchain in the Remainder of This Book 216
Outlook 216
Summary 216
Stage V:Using the Blockchain, Summary, and Outlook 218
Step24: Using the Blockchain 219
The Metaphor 219
Characteristics of the Blockchain 220
Generic Application Patterns 220
Proof of Existence 221
Proof of Nonexistence 221
Proof of Time 221
Proof of Order 221
Proof of Identity 222
Proof of Authorship 222
Proof of Ownership 222
Specific Use Cases 222
Analyzing Blockchain Applications 224
Are the Requirements for Using the Blockchain Fulfilled? 224
What Kind of Blockchain Is Used? 225
What Is the Added Value of Using the Purely Peer-to-Peer System? 226
What Is the Application Idea? 226
What Is the Business Case? 227
How Are Peers Compensated for Contributing Resources to the System and Maintaining Integrity? 227
Outlook 228
Summary 228
Step25: Summarizing and Going Further 230
The Metaphor 231
Further Developments and Alternatives 231
Minor Technical Improvements and Variations 232
Improving Scalability 233
Conceptual Evolutions 233
Access Rights 233
Privacy 234
Consensus 234
Transactions 235
Inventory Data 236
Data Structure 236
Major Accomplishments of the Blockchain 237
Disintermediation 237
Automation 238
Standardization 238
Streamlining Processes 238
Increased Processing Speed 238
Cost Reduction 238
Shift Toward Trust in Protocols and Technology 239
Making Trust a Commodity 239
Increased Technology Awareness 239
Possible Disadvantages 240
Lack of Privacy 240
Loss of Responsibility 240
Loss of Jobs 241
Reintermediation 241
The Future 241
Limited Enthusiast Projects 241
Large-Scale Commercial Usage 242
Governmental Projects 242
Outlook 242
Summary 243
Index 244