Big Data Systems

Jawwad A. Shamsi completed B.E. (Electrical Engineering) from NED University of Enginnering and Technology, Karachi in 1998. He completed his MS in Computer and Information Sciences from University of Michigan-Dearborn, MI, USA in 2002. In 2009, he completed his PhD. from Wayne State University, MI, USA. He has also worked as a Programmar Analyst in USA from 2000 to 2002. In 2009, he joined FAST- National Univesity of Computer and Emerging Sciences (NUCES), Karachi. He has served as the head of computer science department from 2012 to 2017. Currently, he is serving as a Professor of Computer Science and Director of the Karachi Campus. He also leads a research group - syslab (http://syslab.khi.nu.edu.pk). His research is focused on developing systems which can meet the growing needs of scalability, security, high performance, robustness, and agility. His research has been funded by different International and National agencies including NVIDIA and Higher Education Commission, Pakistan. Muhammad Ali Khojaye has more than decade of industrial experience ranging from the cloud-native side of things to distributed systems design, CI/CD, and infrastructure. His current technical interests revolve around big data, cloud, containers, and large scale systems design. He currently lives in the Glasgow suburbs with his wife and son. When he's not at work, Ali enjoys cycling, travelling, and spending time with family and friends.

Preface
Author Bios
Acknowledgements
List of Figures
List of Tables

Introduction to Big Data Systems
1.1 INTRODUCTION: REVIEW OF BIG DATA SYSTEMS
1.2 UNDERSTANDING BIG DATA
1.3 TYPE OF DATA: TRANSACTIONAL OR ANALYTICAL
1.4 REQUIREMENTS AND CHALLENGES OF BIG DATA
1.5 CONCLUDING REMARKS
1.6 FURTHER READING
1.7 EXERCISE QUESTIONS

Architecture and Organization of Big Data Systems
2.1 ARCHITECTURE FOR BIG DATA SYSTEMS
2.2 ORGANIZATION OF BIG DATA SYSTEMS: CLUSTERS
2.3 CLASSIFICATION OF CLUSTERS: DISTRIBUTED MEMORY VS. SHARED MEMORY
2.4 CONCLUDING REMARKS
2.5 FURTHER READING
2.6 EXERCISE QUESTIONS

Cloud Computing for Big Data
3.1 CLOUD COMPUTING
3.2 VIRTUALIZATION
3.3 PROCESSOR VIRTUALIZATION
3.4 CONTAINERIZATION
3.5 VIRTUALIZATION OR CONTAINERIZATION
3.6 FOG COMPUTING
3.7 EXAMPLES
3.8 CONCLUDING REMARKS
3.9 FURTHER READING
3.10 EXERCISE QUESTIONS

HADOOP: An Efficient Platform for Storing and Processing Big Data
4.1 REQUIREMENTS FOR PROCESSING AND STORING BIG DATA
4.2 HADOOP - THE BIG PICTURE
4.3 HADOOP DISTRIBUTED FILE SYSTEM
4.4 MAPREDUCE
4.5 HBASE
4.6 CONCLUDING REMARKS
4.7 FURTHER READING
4.8 EXERCISE QUESTIONS

Enhancements in Hadoop
5.1 ISSUES WITH HADOOP
5.2 YARN
5.3 PIG
5.4 HIVE
5.5 DREMEL
5.6 IMPALA
5.7 DRILL
5.8 DATA TRANSFER
5.9 AMBARI
5.10 CONCLUDING REMARKS
5.11 FURTHER READING
5.12 EXERCISE QUESTIONS

Spark
6.1 LIMITATIONS OF MAPREDUCE
6.2 INTRODUCTION TO SPARK
6.3 SPARK CONCEPTS
6.4 SPARK SQL
6.5 SPARK MLLIB
6.6 STREAM BASED SYSTEM
6.7 SPARK STREAMING
6.8 CONCLUDING REMARKS
6.9 FURTHER READING
6.10 EXERCISE QUESTIONS

NoSQL Systems
7.1 INTRODUCTION
7.2 HANDLING BIG DATA SYSTEMS - PARALLEL RDBMS
7.3 EMERGENCE OF NOSQL SYSTEMS
7.4 KEY-VALUE DATABASE
7.5 DOCUMENT-ORIENTED DATABASE
7.6 COLUMN-ORIENTED DATABASE
7.7 GRAPH DATABASE
7.8 CONCLUDING REMARKS
7.9 FURTHER READING
7.10 EXERCISE QUESTIONS

NewSQL Systems
8.1 INTRODUCTION
8.2 TYPES OF NEWSQL SYSTEMS
8.3 FEATURES
8.4 NEWSQL SYSTEMS: CASE STUDIES
8.5 CONCLUDING REMARKS
8.6 FURTHER READING
8.7 EXERCISE QUESTIONS

Networking for Big Data
9.1 NETWORK ARCHITECTURE FOR BIG DATA SYSTEMS
9.2 CHALLENGES AND REQUIREMENTS
9.3 NETWORK PROGRAMMABILITY AND SOFTWARE DEFINED NETWORKING
9.4 LOW LATENCY AND HIGH SPEED DATA TRANSFER
9.5 AVOIDING TCP INCAST - ACHIEVING LOW LATENCY
AND HIGH THROUGHPUT
9.6 FAULT TOLERANCE
9.7 CONCLUDING REMARKS
9.8 FURTHER READING
9.9 EXERCISE QUESTIONS

Security for Big Data
10.1 INTRODUCTION
10.2 SECURITY REQUIREMENTS
10.3 SECURITY: ATTACK TYPES AND MECHANISMS
10.4 ATTACK DETECTION AND PREVENTION
10.5 CONCLUDING REMARKS
10.6 FURTHER READING
10.7 EXERCISE QUESTIONS

Privacy for Big Data
11.1 INTRODUCTION
11.2 UNDERSTANDING BIG DATA AND PRIVACY
11.3 PRIVACY VIOLATIONS AND THEIR IMPACT
11.4 TYPES OF PRIVACY VIOLATIONS
11.5 PRIVACY PROTECTION SOLUTIONS AND THEIR LIMITATIONS
11.6 CONCLUDING REMARKS
11.7 FURTHER READING
11.8 EXERCISE QUESTIONS

High Performance Computing for Big Data
12.1 INTRODUCTION
12.2 SCALABILITY: NEED FOR HPC
12.3 GRAPHIC PROCESSING UNIT
12.4 TENSOR PROCESSING UNIT
12.5 HIGH SPEED INTERCONNECTS
12.6 MESSAGE PASSING INTERFACE
12.7 OPENMP
12.8 OTHER FRAMEWORKS
12.9 CONCLUDING REMARKS
12.10 FURTHER READING
12.11 EXERCISE QUESTIONS

Deep Learning with Big Data
13.1 INTRODUCTION
13.2 FUNDAMENTALS
13.3 NEURAL NETWORK
13.4 TYPES OF DEEP NEURAL NETWORK
13.5 BIG DATA APPLICATIONS USING DEEP LEARNING
13.6 CONCLUDING REMARKS
13.7 FURTHER READING
13.8 EXERCISE QUESTIONS

Big Data Case Studies
14.1 GOOGLE EARTH ENGINE
14.2 FACEBOOK MESSAGES APPLICATION
14.3 HADOOP FOR REAL-TIME ANALYTICS
14.4 BIG DATA PROCESSING AT UBER
14.5 BIG DATA PROCESSING AT LINKEDIN
14.6 DISTRIBUTED GRAPH PROCESSING AT GOOGLE
14.7 FUTURE TRENDS
14.8 CONCLUDING REMARKS
14.9 FURTHER READING
14.10 EXERCISE QUESTIONS

Bibliography
Index