Mastering Kibana 6.x

Visualize your Elastic Stack data with histograms, maps, charts, and graphs

Anurag Srivastava

• Publisher: Packt Publishing
• Year: 2018
• Print ISBN:
  9781788831031
• Edition: 1
• eBook ISBN:
  9781788834032
• Topic: Computer Science
• Subtopic: Data Processing
• Index: Computer Science

Get to grips with Kibana and its advanced functions to create interactive visualizations and dashboards

Key Features

• Explore visualizations and perform histograms, stats, and map analytics
• Unleash X-Pack and Timelion, and learn alerting, monitoring, and reporting features
• Manage dashboards with Beats and create machine learning jobs for faster analytics

Book Description

Kibana is one of the popular tools among data enthusiasts for slicing and dicing large datasets and uncovering Business Intelligence (BI) with the help of its rich and powerful visualizations.

To begin with, Mastering Kibana 6.x quickly introduces you to the features of Kibana 6.x, before teaching you how to create smart dashboards in no time. You will explore metric analytics and graph exploration, followed by understanding how to quickly customize Kibana dashboards. In addition to this, you will learn advanced analytics such as maps, hits, and list analytics. All this will help you enhance your skills in running and comparing multiple queries and filters, influencing your data visualization skills at scale.

With Kibana's Timelion feature, you can analyze time series data with histograms and stats analytics. By the end of this book, you will have created a speedy machine learning job using X-Pack capabilities.

What you will learn

• Create unique dashboards with various intuitive data visualizations
• Visualize Timelion expressions with added histograms and stats analytics
• Integrate X-Pack with your Elastic Stack in simple steps
• Extract data from Elasticsearch for advanced analysis and anomaly detection using dashboards
• Build dashboards from web applications for application logs
• Create monitoring and alerting dashboards using Beats

Who this book is for

Mastering Kibana 6.x is for you if you are a big data engineer, DevOps engineer, or data scientist aspiring to go beyond data visualization at scale and gain maximum insights from their large datasets. Basic knowledge of Elasticstack will be an added advantage, although not mandatory.

Understanding X-Pack Features

The default Elastic Stack comes with Elasticsearch, Kibana, Logstash, and Beats. We can extend the Elastic Stack setup by installing X-Pack. It has different features, such as providing security, monitoring, reporting, alerting, and graph and machine learning capabilities in an easy-to-install package.

By default, there is no security in Elastic Stack, and through X-Pack, we can enable authentication and define roles and permissions. It monitors the Elastic Stack and provides us with the feature to monitor the application's health seamlessly. Also, we can easily generate PDF reports, using Kibana visualization, after installing X-Pack.

We can seamlessly integrate X-Pack in a running Elastic Stack setup by a simple installation process. The X-Pack security feature provides us with the option to secure Elastic Stack. Using it, we can set password protection, add authentication, filter on IP, or encrypt communications.

In this chapter, we will cover the following topics:

• Installing X-Pack: Here, we will cover the steps to install X-Pack on Elastic Stack. It will include the following steps:
  • Installing X-Pack into Elasticsearch
  • Installing X-Pack into Kibana
• Features of X-Pack: Under this topic, we will cover the main features of X-Pack, as follows:
  • Monitoring: Under monitoring, we will cover Elasticsearch and Kibana monitoring
  • Security: With X-Pack, we have the option to secure our data and manage users with roles
  • Machine learning: Machine learning is basically the use of algorithms and methods on data to get the data-driven predictions, decision making, and modeling
  • Application Performance Monitoring (APM): This is built on top of Elastic Stack to monitor the performance of external applications
  • Logging: We can use the logging feature of X-Pack to ingest logs from different data sources
  • Metrics: Through metrics, we can fetch metrics data from the operating system and running services on the server

Installing X-Pack

In this section, we will take a look at the steps to install X-Pack for Elasticsearch and Kibana. It provides various features, as we have already discussed. It also changes the different views, such as adding the Reporting tab under Visualization and the Query Profile under Dev Tools. Similarly, there are many additions, which we may notice after installing X-Pack. Now let's take a look at how to install X-Pack.

Installing X-Pack into Elasticsearch

To install X-Pack to our Elasticsearch setup, we will need to perform following steps:

1. First, we need to stop Elasticsearch.
2. Move to the Elasticsearch setup directory and then enter the following command:

 bin/elasticsearch-plugin install x-pack

3. After successfully running the command, start Elasticsearch.
   1. Generate default passwords, as follows:

 bin/x-pack/setup-passwords auto

The preceding command will create the passwords automatically, which we will need to copy to log into the system.

Installing X-Pack into Kibana

To install X-Pack into our Kibana setup, we need to perform the following steps:

1. First, we need to stop Kibana. To stop Kibana on Ubuntu, run the following command:

sudo service kibana stop

2. Move to the Kibana installation directory and then enter the following command:

bin/kibana-plugin install x-pack

3. We will need to add the following credentials to the kibana.yml file:

elasticsearch.username: "kibana"
elasticsearch.password: "<pwd>

4. Start Kibana, after successfully installing X-Pack for Kibana. To start Kibana on Ubuntu, run the following command:

sudo service kibana start

5. After that, we will need to navigate to Kibana by entering the following URL on our browser:

http://localhost:5601

6. Now, we will need to Log in through the elastic user, with the created password:

The preceding screenshot shows us the login page on Kibana after installing X-Pack. Here, we can log in with the created credentials, during X-Pack installation. The username is elastic and is the admin user, with which we can access all features on X-Pack.

Features of X-Pack

In this section, we will discuss the main features of X-Pack. The main feature, which we will discuss first, is monitoring (from the introduction section), where we will discuss how to monitor Elasticsearch and Kibana. Then, we will discuss the security feature of X-Pack using which we can create users and roles, and based on the assigned roles, a user can access different sections of Kibana. Machine learning is a very prominent feature of X-Pack, which was introduced in Version 5.4 of Kibana. Machine learning in Kibana uses algorithms and methods on data to get the data-driven predictions, decision making, and modeling. We will also discuss APM, which is built on top of Elastic Stack, to monitor the performance of external applications. The next important feature is logging, which is used to ingest logs from different data sou...

Table of contents

• Title Page
• Copyright and Credits
• Packt Upsell
• Contributors
• Preface
• Revising the ELK Stack
• Setting Up and Customizing the Kibana Dashboard
• Exploring Your Data
• Visualizing the Data
• Dashboarding to Showcase Key Performance Indicators
• Handling Time Series Data with Timelion
• Interact with Your Data Using Dev Tools
• Tweaking Your Configuration with Kibana Management
• Understanding X-Pack Features
• Machine Learning with Kibana
• Create Super Cool Dashboard from a Web Application
• Different Use Cases of Kibana
• Creating Monitoring Dashboards Using Beats
• Best Practices
• Other Books You May Enjoy