Big Data Analytics for Internet of Things

Tausifa Jan Saleem is currently pursuing her Doctor of Philosophy (Ph.D) from National Institute of Technology Srinagar, India. She has received the Bachelor of Technology (B. Tech.) degree in Information Technology (IT) from National Institute of Technology Srinagar, India and the M.Tech. degree in Computer Science from University of Jammu, India. She has published more than 10 research articles in reputed journals (indexed by Scopus and SCI) and conferences (indexed by Scopus). Her research areas of interest include Internet of Things, Data Analytics, Machine Learning, and Deep Learning. Mohammad Ahsan Chishti, Ph.D, is Dean at the School of Engineering & Technology and Associate Professor in the Department of Information Technology at the Central University of Kashmir. He has published over 100 scholarly papers and holds 12 patents. He is the recipient of “Young Engineers Award 2015-2016” from IEI and “Young Scientist Award 2009-2010” from the government of Jammu and Kashmir. He is a Senior Member of the IEEE, MIEI, MCSI & MIETE.

List of Contributors xv

List of Abbreviations xix

1 Big Data Analytics for the Internet of Things: An Overview 1
Tausifa Jan Saleem and Mohammad Ahsan Chishti

2 Data, Analytics and Interoperability Between Systems (IoT) is Incongruous with the Economics of Technology: Evolution of Porous Pareto Partition (P3) 7
Shoumen Palit Austin Datta, Tausifa Jan Saleem, Molood Barati, María Victoria López López, Marie-Laure Furgala, Diana C. Vanegas, Gérald Santucci, Pramod P. Khargonekar, and Eric S. McLamore

2.1 Context 8

2.2 Models in the Background 12

2.3 Problem Space: Are We Asking the Correct Questions? 14

2.4 Solutions Approach: The Elusive Quest to Build Bridges Between Data and Decisions 15

2.5 Avoid This Space: The Deception Space 17

2.6 Explore the Solution Space: Necessary to Ask Questions That May Not Have Answers, Yet 17

2.7 Solution Economy: Will We Ever Get There? 19

2.8 Is This Faux Naïveté in Its Purest Distillate? 21

2.9 Reality Check: Data Fusion 22

2.10 “Double A” Perspective of Data and Tools vs. The Hypothetical Porous Pareto (80/20) Partition 28

2.11 Conundrums 29

2.12 Stigma of Partition vs. Astigmatism of Vision 38

2.13 The Illusion of Data, Delusion of Big Data, and the Absence of Intelligence in AI 40

2.14 In Service of Society 50

2.15 Data Science in Service of Society: Knowledge and Performance from PEAS 52

2.16 Temporary Conclusion 60

Acknowledgements 63

References 63

3 Machine Learning Techniques for IoT Data Analytics 89
Nailah Afshan and Ranjeet Kumar Rout

3.1 Introduction 89

3.2 Taxonomy of Machine Learning Techniques 94

3.2.1 Supervised ML Algorithm 95

3.2.1.1 Classification 96

3.2.1.2 Regression Analysis 98

3.2.1.3 Classification and Regression Tasks 99

3.2.2 Unsupervised Machine Learning Algorithms 103

3.2.2.1 Clustering 103

3.2.2.2 Feature Extraction 106

3.2.3 Conclusion 107

References 107

4 IoT Data Analytics Using Cloud Computing 115
Anjum Sheikh, Sunil Kumar, and Asha Ambhaikar

4.1 Introduction 115

4.2 IoT Data Analytics 117

4.2.1 Process of IoT Analytics 117

4.2.2 Types of Analytics 118

4.3 Cloud Computing for IoT 118

4.3.1 Deployment Models for Cloud 120

4.3.1.1 Private Cloud 120

4.3.1.2 Public Cloud 120

4.3.1.3 Hybrid Cloud 121

4.3.1.4 Community Cloud 121

4.3.2 Service Models for Cloud Computing 122

4.3.2.1 Software as a Service (SaaS) 122

4.3.2.2 Platform as a Service (PaaS) 122

4.3.2.3 Infrastructure as a Service (IaaS) 122

4.3.3 Data Analytics on Cloud 123

4.4 Cloud-Based IoT Data Analytics Platform 123

4.4.1 Atos Codex 125

4.4.2 AWS IoT 125

4.4.3 IBM Watson IoT 126

4.4.4 Hitachi Vantara Pentaho, Lumada 127

4.4.5 Microsoft Azure IoT 128

4.4.6 Oracle IoT Cloud Services 129

4.5 Machine Learning for IoT Analytics in Cloud 132

4.5.1 ML Algorithms for Data Analytics 132

4.5.2 Types of Predictions Supported by ML and Cloud 136

4.6 Challenges for Analytics Using Cloud 137

4.7 Conclusion 139

References 139

5 Deep Learning Architectures for IoT Data Analytics 143
Snowber Mushtaq and Omkar Singh

5.1 Introduction 143

5.1.1 Types of Learning Algorithms 146

5.1.1.1 Supervised Learning 146

5.1.1.2 Unsupervised Learning 146

5.1.1.3 Semi-Supervised Learning 146

5.1.1.4 Reinforcement Learning 146

5.1.2 Steps Involved in Solving a Problem 146

5.1.2.1 Basic Terminology 147

5.1.2.2 Training Process 147

5.1.3 Modeling in Data Science 147

5.1.3.1 Generative 148

5.1.3.2 Discriminative 148

5.1.4 Why DL and IoT? 148

5.2 DL Architectures 149

5.2.1 Restricted Boltzmann Machine 149

5.2.1.1 Training Boltzmann Machine 150

5.2.1.2 Applications of RBM 151

5.2.2 Deep Belief Networks (DBN) 151

5.2.2.1 Training DBN 152

5.2.2.2 Applications of DBN 153

5.2.3 Autoencoders 153

5.2.3.1 Training of AE 153

5.2.3.2 Applications of AE 154

5.2.4 Convolutional Neural Networks (CNN) 154

5.2.4.1 Layers of CNN 155

5.2.4.2 Activation Functions Used in CNN 156

5.2.4.3 Applications of CNN 158

5.2.5 Generative Adversarial Network (GANs) 158

5.2.5.1 Training of GANs 158

5.2.5.2 Variants of GANs 159

5.2.5.3 Applications of GANs 159

5.2.6 Recurrent Neural Networks (RNN) 159

5.2.6.1 Training of RNN 160

5.2.6.2 Applications of RNN 161

5.2.7 Long Short-Term Memory (LSTM) 161

5.2.7.1 Training of LSTM 161

5.2.7.2 Applications of LSTM 162

5.3 Conclusion 162

References 163

6 Adding Personal Touches to IoT: A User-Centric IoT Architecture 167
Sarabjeet Kaur Kochhar

6.1 Introduction 167

6.2 Enabling Technologies for BDA of IoT Systems 169

6.3 Personalizing the IoT 171

6.3.1 Personalization for Business 172

6.3.2 Personalization for Marketing 172

6.3.3 Personalization for Product Improvement and Service Optimization 173

6.3.4 Personalization for Automated Recommendations 174

6.3.5 Personalization for Improved User Experience 174

6.4 Related Work 175

6.5 User Sensitized IoT Architecture 176

6.6 The Tweaked Data Layer 178

6.7 The Personalization Layer 180

6.7.1 The Characterization Engine 180

6.7.2 The Sentiment Analyzer 182

6.8 Concerns and Future Directions 183

6.9 Conclusions 184

References 185

7 Smart Cities and the Internet of Things 187
Hemant Garg, Sushil Gupta, and Basant Garg

7.1 Introduction 187

7.2 Development of Smart Cities and the IoT 188

7.3 The Combination of the IoT with Development of City Architecture to Form Smart Cities 189

7.3.1 Unification of the IoT 190

7.3.2 Security of Smart Cities 190

7.3.3 Management of Water and Related Amenities 190

7.3.4 Power Distribution and Management 191

7.3.5 Revenue Collection and Administration 191

7.3.6 Management of City Assets and Human Resources 192

7.3.7 Environmental Pollution Management 192

7.4 How Future Smart Cities Can Improve Their Utilization of the Internet of All Things, with Examples 193

7.5 Conclusion 194

References 195

8 A Roadmap for Application of IoT-Generated Big Data in Environmental Sustainability 197
Ankur Kashyap

8.1 Background and Motivation 197

8.2 Execution of the Study 198

8.2.1 Role of Big Data in Sustainability 198

8.2.2 Present Status and Future Possibilities of IoT in Environmental Sustainability 199

8.3 Proposed Roadmap 202

8.4 Identification and Prioritizing the Barriers in the Process 204

8.4.1 Internet Infrastructure 204

8.4.2 High Hardware and Software Cost 204

8.4.3 Less Qualified Workforce 204

8.5 Conclusion and Discussion 205

References 205

9 Application of High-Performance Computing in Synchrophasor Data Management and Analysis for Power Grids 209
C.M. Thasnimol and R. Rajathy

9.1 Introduction 209

9.2 Applications of Synchrophasor Data 210

9.2.1 Voltage Stability Analysis 211

9.2.2 Transient Stability 212

9.2.3 Out of Step Splitting Protection 213

9.2.4 Multiple Event Detection 213

9.2.5 State Estimation 213

9.2.6 Fault Detection 214

9.2.7 Loss of Main (LOM) Detection 214

9.2.8 Topology Update Detection 214

9.2.9 Oscillation Detection 215

9.3 Utility Big Data Issues Related to PMU-Driven Applications 215

9.3.1 Heterogeneous Measurement Integration 215

9.3.2 Variety and Interoperability 216

9.3.3 Volume and Velocity 216

9.3.4 Data Quality and Security 216

9.3.5 Utilization and Analytics 217

9.3.6 Visualization of Data 218

9.4 Big Data Analytics Platforms for PMU Data Processing 219

9.4.1 Hadoop 220

9.4.2 Apache Spark 221

9.4.3 Apache HBase 222

9.4.4 Apache Storm 222

9.4.5 Cloud-Based Platforms 223

9.5 Conclusions 224

References 224

10 Intelligent Enterprise-Level Big Data Analytics for Modeling and Management in Smart Internet of Roads 231
Amin Fadaeddini, Babak Majidi, and Mohammad Eshghi

10.1 Introduction 231

10.2 Fully Convolutional Deep Neural Network for Autonomous Vehicle Identification 233

10.2.1 Detection of the Bounding Box of the License Plate 233

10.2.2 Segmentation Objective 234

10.2.3 Spatial Invariances 234

10.2.4 Model Framework 234

10.2.4.1 Increasing the Layer of Transformation 234

10.2.4.2 Data Format of Sample Images 235

10.2.4.3 Applying Batch Normalization 236

10.2.4.4 Network Architecture 236

10.2.5 Role of Data 236

10.2.6 Synthesizing Samples 236

10.2.7 Invariances 237

10.2.8 Reducing Number of Features 237

10.2.9 Choosing Number of Classes 238

10.3 Experimental Setup and Results 239

10.3.1 Sparse Softmax Loss 239

10.3.2 Mean Intersection Over Union 240

10.4 Practical Implementation of Enterprise-Level Big Data Analytics for Smart City 240

10.5 Conclusion 244

References 244

11 Predictive Analysis of Intelligent Sensing and Cloud-Based Integrated Water Management System 247
Tanuja Patgar and Ripal Patel

11.1 Introduction 247

11.2 Literature Survey 248

11.3 Proposed Six-Tier Data Framework 250

11.3.1 Primary Components 251

11.3.2 Contact Unit (FC-37) 253

11.3.3 Internet of Things Communicator (ESP8266) 253

11.3.4 GSM-Based ARM and Control System 253

11.3.5 Methodology 253

11.3.6 Proposed Algorithm 256

11.4 Implementation and Result Analysis 257

11.4.1 Water Report for Home 1 and Home 2 Modules 263

11.5 Conclusion 263

References 263

12 Data Security in the Internet of Things: Challenges and Opportunities 265
Shashwati Banerjea, Shashank Srivastava, and Sachin Kumar

12.1 Introduction 265

12.2 IoT: Brief Introduction 266

12.2.1 Challenges in a Secure IoT 267

12.2.2 Security Requirements in IoT Architecture 268

12.2.2.1 Sensing Layer 268

12.2.2.2 Network Layer 269

12.2.2.3 Interface Layer 271

12.2.3 Common Attacks in IoT 271

12.3 IoT Security Classification 272

12.3.1 Application Domain 272

12.3.1.1 Authentication 272

12.3.1.2 Authorization 274

12.3.1.3 Depletion of Resources 274

12.3.1.4 Establishment of Trust 275

12.3.2 Architectural Domain 275

12.3.2.1 Authentication in IoT Architecture 275

12.3.2.2 Authorization in IoT Architecture 276

12.3.3 Communication Channel 276

12.4 Security in IoT Data 277

12.4.1 IoT Data Security: Requirements 277

12.4.1.1 Data: Confidentiality, Integrity, and Authentication 278

12.4.1.2 Data Privacy 279

12.4.2 IoT Data Security: Research Directions 280

12.5 Conclusion 280

References 281

13 DDoS Attacks: Tools, Mitigation Approaches, and Probable Impact on Private Cloud Environment 285
R. K. Deka, D. K. Bhattacharyya, and J. K. Kalita

13.1 Introduction 285

13.1.1 State of the Art 287

13.1.2 Contribution 288

13.1.3 Organization 290

13.2 Cloud and DDoS Attack 290

13.2.1 Cloud Deployment Models 290

13.2.1.1 Differences Between Private Cloud and Public Cloud 293

13.2.2 DDoS Attacks 294

13.2.2.1 Attacks on Infrastructure Level 294

13.2.2.2 Attacks on Application Level 296

13.2.3 DoS/DDoS Attack on Cloud: Probable Impact 297

13.3 Mitigation Approaches 298

13.3.1 Discussion 309

13.4 Challenges and Issues with Recommendations 309

13.5 A Generic Framework 310

13.6 Conclusion and Future Work 312

References 312

14 Securing the Defense Data for Making Better Decisions Using Data Fusion 321
Syed Rameem Zahra

14.1 Introduction 321

14.2 Analysis of Big Data 322

14.2.1 Existing IoT Big Data Analytics Systems 322

14.2.2 Big Data Analytical Methods 324

14.2.3 Challenges in IoT Big Data Analytics 324

14.3 Data Fusion 325

14.3.1 Opportunities Provided by Data Fusion 326

14.3.2 Data Fusion Challenges 326

14.3.3 Stages at Which Data Fusion Can Happen 326

14.3.4 Mathematical Methods for Data Fusion 326

14.4 Data Fusion for IoT Security 327

14.4.1 Defense Use Case 329

14.5 Conclusion 329

References 330

15 New Age Journalism and Big Data (Understanding Big Data and Its Influence on Journalism) 333
Asif Khan and Heeba Din

15.1 Introduction 333

15.1.1 Big Data Journalism: The Next Big Thing 334

15.1.2 All About Data 336

15.1.3 Accessing Data for Journalism 337

15.1.4 Data Analytics: Tools for Journalists 338

15.1.5 Case Studies – Big Data 340

15.1.5.1 BBC Big Data 340

15.1.5.2 The Guardian Data Blog 342

15.1.5.3 Wikileaks 344

15.1.5.4 World Economic Forum 344

15.1.6 Big Data – Indian Scenario 345

15.1.7 Internet of Things and Journalism 346

15.1.8 Impact on Media/Journalism 347

References 348

16 Two Decades of Big Data in Finance: Systematic Literature Review and Future Research Agenda 351
Nufazil Altaf

16.1 Introduction 351

16.2 Methodology 353

16.3 Article Identification and Selection 353

16.4 Description and Classification of Literature 354

16.4.1 Research Method Employed 354

16.4.2 Articles Published Year Wise 355

16.4.3 Journal of Publication 356

16.5 Content and Citation Analysis of Articles 356

16.5.1 Citation Analysis 356

16.5.2 Content Analysis 357

16.5.2.1 Big Data in Financial Markets 358

16.5.2.2 Big Data in Internet Finance 359

16.5.2.3 Big Data in Financial Services 359

16.5.2.4 Big Data and Other Financial Issues 360

16.6 Reporting of Findings and Research Gaps 360

16.6.1 Findings from the Literature Review 361

16.6.1.1 Lack of Symmetry 361

16.6.1.2 Dominance of Research on Financial Markets, Internet Finance, and Financial Services 361

16.6.1.3 Dominance of Empirical Research 361

16.6.2 Directions for Future Research 362

References 362

Index 367