Practical Elk Stack

Gurpreet S. Sachdeva is a Technology Leader with 20 years of experience working on some of the most challenging technologies related to Communication Software, Enterprise Computing and Cloud Computing. Gurpreet did his B. Tech (Computer Engineering) from NIT, Kurukshetra and M.S. (Software Systems) from BITS, Pilani. He is currently working as Director - Technology with Aricent, Gurgaon. He is a keen Java enthusiast and co-founder of Delhi - NCR - Java User Group. Gurpreet is passionate about building cloud scale software and manage it through ELK stack along with other DevOps tools. Gurpreet is an invited speaker in prestigious conferences like Oracle - Java One, Great India Developer Summit, Indic Threads. Gurpreet blogs at http://www.thistechnologylife.com

Chapter 1: Introduction to ELK StackChapter Goal: This chapter emphasizes the importance of log analysis in today's big data crazy world. It would go on to analyze the challenges with log analysis. It presents ELK stack as a thorough solution for log analysis. Different components of ELK Stack - Elasticsearch, Logstash and Kibana are introduced with a description of their functions and installation.No of pages: 25Sub -Topics1.Log Analysis in Today's World2.The ELK Stack3.ELK Data Pipeline4.ELK Stack Installation" Chapter 2: Shipping, Filtering and Parsing Events with LogstashChapter Goal: The goal of this chapter is to get started with using Logstash for log generation, collection and filtering. It starts with introducing configuration settings of Logstash. It then goes on to illustrate how Logstash facilitates shipping of logs, filtering and transforming any type of data to a common format. This can further help in arriving at actionable insights.No of pages: 40Sub - Topics1.Configuring Logstash2.Shipping Events3.Filtering Events with Logstash4.Outputting Events from Logstash" "div>Chapter 3: Extending LogstashChapter Goal: The goal of this chapter is to illustrate how Logstash is internally organized using Plugins. Logstash has a diverse collection of input, filter, codec and output plugins. An overview of the common plugins would be provided. It would then show how to create and use your own custom plugin.No of pages: 20Sub - Topics: 1.Plugin Management2.Structure of a Plugin3.Adding custom plugins" Chapter 4: Creating, Indexing and Deleting DataChapter Goal: This chapter introduces data management using Elasticsearch. It covers features of Elasticsearch. Data is organized as documents. This chapter would show how to add data, index it, update it, and delete it. It also goes on to show how to work with distributed document stores.No of pages: 40Sub - Topics: 1.Anatomy of a Document2.Creating Document3.Indexing a Document4.Updating a Document5.Deleting a Document6.Distributed Document Store Chapter 5: Searching DataChapter Goal: The goal of this chapter is to explore the elaborate mechanism for searching data available in Elasticsearch. It covers both the search query variations - Search Lite and Full Body Search. Then it illustrates Query DSL and Filters.No of pages: 25Sub - Topics: 1.Basic Search2.Search with Multi-Index, Multi-Type3.Pagination in Search4.Search Lite5.Query DSL6.Queries and Filters7.Advanced Search Concepts" Chapter 6: Mapping and AnalysisChapter Goal: The goal of this chapter is to examine how Elasticsearch maps data. It then goes on to show how to map data for relevant analysis.No of pages: 25Sub - Topics: 1.Data Type2.Analyzers3.Mapping4.Composite Field Types Chapter 7: Data Exploration with AggregatesChapter Goal: This chapter explores the subject of Aggregates. It would help in giving a top level view of entire set of documents. This is unlike queries which just focus on a particular document. It is also shown how to group documents into buckets.No of pages: 20Sub - Topics: 1.Buckets and Metrics2.Bar Charts3.Scoping Aggregations4.Aggregates with Filters5.Approximate Aggregates Chapter 8: Exploring KibanaChapter Goal: This chapter introduces Kibana. It explains basic concepts and key features.No of pages: 25Sub - Topics: 1.Kibana Key Concepts2.Kibana Features Chapter 9: Kibana - Discover, Visualize and DashboardChapter Goal: This chapter will show how to work with Kibana by illustrating its interface to filter and visualize log messages gathered by Elasticsearch. It will cover the main interface components, and demonstrate how to create searches, visualizations, and dashboards.No of pages: 40Sub - Topics: 1.Exploring Discover Page2.Exploring Visualize Page3.Exploring Dashboard Page4.Settings Page Chapter 10: Insights with ELK StackChapter Goal: This chapter really ties up all the components of ELK stack together to arrive at actionable insights. It first shows how to do proper data modelling so that useful logs are emitted by Logstash. Then these logs need to be organized into documents with proper indexing. Finally, appropriate dashboards need to be configured in Kibana to provide data analytics.No of pages: 30Sub - Topics: 1.Data Modelling2.Configuring Logstash input3.Analysis with Kibana Chapter 11: Designing for ScaleChapter Goal: Elasticsearch can be used to index and search petabytes of data. This chapter shows how Elasticsearch can be run in a cluster containing hundreds of nodes. This requires planning and design. It would cover replica shards and multiple indices.No of pages: 25Sub - Topics: 1.Elasticsearch Cluster2.Logstash Second Indexer3.Replica Shards4.Multiple Indices Chapter 12: ELK Stack at WorkChapter Goal: This chapter would cover practical areas where ELK stack is being used. All the earlier chapters were laying the foundation for the practical aspects of ELK stack. It is being used in companies like Facebook, Netflix. It can be used for lot of diverse purpose like Social Media Analysis, Troubleshooting applications or Grid Monitoring at CERN.No of pages: 40Sub - Topics: 1.Social Media Analysis2.Live Application Troubleshooting3.ELK Stack at Github4.ELK Stack at http://stackoverflow.com/5.Real Time Analytics6.Grid Monitoring at CERN7.Searching New York Times Articles8.Message Analytics at Netflix9.Search at Facebook 13: Cover troubleshooting, performance improvement