Blockchain Data Analytics For Dummies

Michael G. Solomon, PhD, is a professor at the University of the Cumberlands who specializes in courses on blockchain and distributed computing systems as well as computer security. He holds numerous security and project management certifications and has written several books on security and project management, including Ethereum For Dummies.

Introduction 1

About This Book 1

Foolish Assumptions 2

Icons Used in This Book 2

Beyond the Book 2

Where to Go from Here 3

Part 1: Intro to Analytics and Blockchain 5

Chapter 1: Driving Business with Data and Analytics 7

Deriving Value from Data 8

Monetizing data 8

Exchanging data 9

Verifying data 10

Understanding and Satisfying Regulatory Requirements 11

Classifying individuals 11

Identifying criminals 11

Examining common privacy laws 12

Predicting Future Outcomes with Data 13

Classifying entities 13

Predicting behavior 14

Making decisions based on models 16

Changing Business Practices to Create Desired Outcomes 16

Defining the desired outcome 17

Building models for simulation 17

Aligning operations and assessing results 18

Chapter 2: Digging into Blockchain Technology 19

Exploring the Blockchain Landscape 20

Managing ownership transfer 20

Doing more with blockchain 21

Understanding blockchain technology 21

Reviewing blockchain’s family tree 22

Fitting blockchain into today’s businesses 25

Understanding Primary Blockchain Types 27

Categorizing blockchain implementations 27

Describing basic blockchain type features 29

Contrasting popular enterprise blockchain implementations 30

Aligning Blockchain Features with Business Requirements 31

Reviewing blockchain core features 31

Examining primary common business requirements 33

Matching blockchain features to business requirements 34

Examining Blockchain Use Cases 35

Managing physical items in cyberspace 35

Handling sensitive information 36

Conducting financial transactions 37

Chapter 3: Identifying Blockchain Data with Value 39

Exploring Blockchain Data 40

Understanding what’s stored in blockchain blocks 40

Recording transaction data 41

Dissecting the parts of a block 43

Decoding block data 47

Categorizing Common Data in a Blockchain 49

Serializing transaction data 49

Logging events on the blockchain 50

Storing value with smart contracts 52

Examining Types of Blockchain Data for Value 52

Exploring basic transaction data 53

Associating real-world meaning to events 53

Aligning Blockchain Data with Real-World Processes 54

Understanding smart contract functions 55

Assessing smart contract event logs 55

Ranking transaction and event data by its effect 55

Chapter 4: Implementing Blockchain Analytics in Business 57

Aligning Analytics with Business Goals 58

Leveraging newly accessible decentralized tools 58

Monetizing data 59

Exchanging and integrating data effectively 59

Surveying Options for Your Analytics Lab 60

Installing the Blockchain Client 61

Installing the Test Blockchain 65

Installing the Testing Environment 68

Getting ready to install Truffle 69

Downloading and installing Truffle 72

Installing the IDE 74

Chapter 5: Interacting with Blockchain Data 79

Exploring the Blockchain Analytics Ecosystem 80

Reviewing your blockchain lab 80

Identifying analytics client options 81

Choosing the best blockchain analytics client 83

Adding Anaconda and Web3.js to Your Lab 84

Verifying platform prerequisites 84

Installing the Anaconda platform 86

Installing the Web3.py library 89

Setting up your blockchain analytics project 90

Writing a Python Script to Access a Blockchain 92

Interfacing with smart contracts 93

Finding a smart contract’s ABI 94

Building a Local Blockchain to Analyze 100

Connecting to your blockchain 101

Invoking smart contract functions 101

Fetching blockchain data 102

Part 2: Fetching Blockchain Chain 105

Chapter 6: Parsing Blockchain Data and Building the Analysis Dataset 107

Comparing On-Chain and External Analysis Options 108

Considering access speed 108

Comparing one-off versus repeated analysis 109

Assessing data completeness 110

Integrating External Data 111

Determining what data you need 112

Extending identities to off-chain data 113

Finding external data 114

Identifying Features 115

Describing how features affect outcomes 116

Comparing filtering and wrapping methods 116

Building an Analysis Dataset 117

Connecting to multiple data sources 118

Building a cross-referenced dataset 118

Cleaning your data 118

Chapter 7: Building Basic Blockchain Analysis Models 121

Identifying Related Data 122

Grouping data based on features (attributes) 123

Determining group membership 126

Discovering relationships among items 129

Making Predictions of Future Outcomes 130

Selecting features that affect outcome 131

Beating the best guess 133

Building confidence 134

Analyzing Time-Series Data 135

Exploring growth and maturity 137

Identifying seasonal trends 138

Describing cycles of results 138

Chapter 8: Leveraging Advanced Blockchain Analysis Models 139

Identifying Participation Incentive Mechanisms 140

Complying with mandates 141

Playing games with partners 141

Rewarding and punishing participants 142

Managing Deployment and Maintenance Costs 143

Lowering the cost of admission 143

Leveraging participation value 145

Aligning ROI with analytics currency 146

Collaborating to Create Better Models 147

Collecting data from a cohort 148

Building models collaboratively 148

Assessing model quality as a team 149

Part 3: Analyzing and Visualizing Blockchain Analysis Data 151

Chapter 9: Identifying Clustered and Related Data 153

Analyzing Data Clustering Using Popular Models 154

Delivering valuable knowledge with cluster analysis 154

Examining popular clustering techniques 155

Understanding k-means analysis 155

Evaluating model effectiveness with diagnostics 160

Implementing Blockchain Data Clustering Algorithms in Python 160

Discovering Association Rules in Data 163

Delivering valuable knowledge with association rules analysis 163

Describing the apriori association rules algorithm 164

Evaluating model effectiveness with diagnostics 167

Determining When to Use Clustering and Association Rules 168

Chapter 10: Classifying Blockchain Data 171

Analyzing Data Classification Using Popular Models 172

Delivering valuable knowledge with classification analysis 172

Examining popular classification techniques 173

Understanding how the decision tree algorithm works 173

Understanding how the naïve Bayes algorithm works 176

Evaluating model effectiveness with diagnostics 178

Implementing Blockchain Classification Algorithms in Python 179

Defining model input data requirements 179

Building your classification model dataset 181

Developing your classification model code 184

Determining When Classification Fits Your Analytics Needs 188

Chapter 11: Predicting the Future with Regression 189

Analyzing Predictions and Relationships Using Popular Models 190

Delivering valuable knowledge with regression analysis 190

Examining popular regression techniques 191

Describing how linear regression works 195

Describing how logistic regression works 198

Evaluating model effectiveness with diagnostics 201

Implementing Regression Algorithms in Python 203

Defining model input data requirements 203

Building your regression model dataset 203

Developing your regression model code 204

Determining When Regression Fits Your Analytics Needs 207

Chapter 12: Analyzing Blockchain Data over Time 209

Analyzing Time Series Data Using Popular Models 210

Delivering valuable knowledge with time series analysis 211

Examining popular time series techniques 211

Visualizing time series results 214

Implementing Time Series Algorithms in Python 216

Defining model input data requirements 217

Developing your time series model code 219

Determining When Time Series Fits Your Analytics Needs 221

Part 4: Implementing Blockchain Analysis Models 223

Chapter 13: Writing Models from Scratch 225

Interacting with Blockchains 226

Connecting to a Blockchain 226

Using an application programming interface to interact with a blockchain 228

Reading from a blockchain 230

Updating previously read blockchain data 234

Examining Blockchain Client Languages and Approaches 236

Introducing popular blockchain client programming languages 237

Comparing popular language pros and cons 238

Deciding on the right language 238

Chapter 14: Calling on Existing Frameworks 239

Benefitting from Standardization 240

Easing the burden of compliance 240

Avoiding inefficient code 242

Raising the bar on quality 244

Focusing on Analytics, Not Utilities 245

Avoiding feature bloat 245

Setting granular goals 246

Managing post-operational models 247

Leveraging the Efforts of Others 248

Deciding between make or buy 248

Scoping your testing efforts 249

Aligning personnel expertise with tasks 250

Chapter 15: Using Third-Party Toolsets and Frameworks 251

Surveying Toolsets and Frameworks 252

Describing TensorFlow 253

Examining Keras 255

Looking at PyTorch 256

Supercharging PyTorch with fast.ai 258

Presenting Apache MXNet 260

Introducing Caffe 261

Describing Deeplearning4j 262

Comparing Toolsets and Frameworks 264

Chapter 16: Putting It All Together 267

Assessing Your Analytics Needs 268

Describing the project’s purpose 268

Defining the process 270

Taking inventory of resources 271

Choosing the Best Fit 273

Understanding personnel skills and affinity 273

Leveraging infrastructure 275

Integrating into organizational culture 276

Embracing iteration 276

Managing the Blockchain Project 277

Part 5: The Part of Tens 279

Chapter 17: Ten Tools for Developing Blockchain Analytics Models 281

Developing Analytics Models with Anaconda 282

Writing Code in Visual Studio Code 283

Prototyping Analytics Models with Jupyter 284

Developing Models in the R Language with RStudio 285

Interacting with Blockchain Data with web3.py 287

Extract Blockchain Data to a Database 288

Extracting blockchain data with EthereumDB 288

Storing blockchain data in a database using Ethereum-etl 288

Accessing Ethereum Networks at Scale with Infura 289

Analyzing Very Large Datasets in Python with Vaex 290

Examining Blockchain Data 291

Exploring Ethereum with Etherscan.io 291

Perusing multiple blockchains with Blockchain.com 292

Viewing cryptocurrency details with ColossusXT 293

Preserving Privacy in Blockchain Analytics with MADANA 293

Chapter 18: Ten Tips for Visualizing Data 295

Checking the Landscape around You 296

Leveraging the Community 297

Making Friends with Network Visualizations 298

Recognizing Subjectivity 299

Using Scale, Text, and the Information You Need 300

Considering Frequent Updates for Volatile Blockchain Data 301

Getting Ready for Big Data 302

Protecting Privacy 302

Telling Your Story 303

Challenging Yourself! 303

Chapter 19: Ten Uses for Blockchain Analytics 305

Accessing Public Financial Transaction Data 306

Connecting with the Internet of Things (IoT) 307

Ensuring Data and Document Authenticity 308

Controlling Secure Document Integrity 308

Tracking Supply Chain Items 310

Empowering Predictive Analytics 310

Analyzing Real-Time Data 311

Supercharging Business Strategy 312

Managing Data Sharing 312

Standardizing Collaboration Forms 312

Index 315