Monitoring Elasticsearch
eBook - ePub

Monitoring Elasticsearch

Dan Noble

Share book
  1. 180 pages
  2. English
  3. ePUB (mobile friendly)
  4. Available on iOS & Android
eBook - ePub

Monitoring Elasticsearch

Dan Noble

Book details
Book preview
Table of contents
Citations

About This Book

Monitor your Elasticsearch cluster's health, and diagnose and solve its performance and reliability issues

About This Book

  • Understand common performance and reliability pitfalls in ElasticSearch
  • Use popular monitoring tools such as ElasticSearch-head, BigDesk, Marvel, Kibana, and more
  • This is a step-by-step guide with lots of case studies on solving real-world ElasticSearch cluster issues

Who This Book Is For

This book is for developers and system administrators who use ElasticSearch in a wide range of capacities. Prior knowledge of ElasticSearch and related technologies would be helpful, but is not necessary.

What You Will Learn

  • Explore your cluster with ElasticSearch-head and BigDesk
  • Access the underlying data of the ElasticSearch monitoring plugins using the ElasticSearch API
  • Analyze your cluster's performance with Marvel
  • Troubleshoot some of the common performance and reliability issues that come up when using ElasticSearch
  • Analyze a cluster's historical performance, and get to the bottom of and recover from system failures
  • Use and install various other tools and plugins such as Kibana and Kopf, which is helpful to monitor ElasticSearch

In Detail

ElasticSearch is a distributed search server similar to Apache Solr with a focus on large datasets, a schema-less setup, and high availability. This schema-free architecture allows ElasticSearch to index and search unstructured content, making it perfectly suited for both small projects and large big data warehouses with petabytes of unstructured data.

This book is your toolkit to teach you how to keep your cluster in good health, and show you how to diagnose and treat unexpected issues along the way. You will start by getting introduced to ElasticSearch, and look at some common performance issues that pop up when using the system. You will then see how to install and configure ElasticSearch and the ElasticSearch monitoring plugins. Then, you will proceed to install and use the Marvel dashboard to monitor ElasticSearch. You will find out how to troubleshoot some of the common performance and reliability issues that come up when using ElasticSearch. Finally, you will analyze your cluster's historical performance, and get to know how to get to the bottom of and recover from system failures.

This book will guide you through several monitoring tools, and utilizes real-world cases and dilemmas faced when using ElasticSearch, showing you how to solve them simply, quickly, and cleanly.

Style and approach

This is a step-by-step guide to monitoring your ElasticSearch cluster and correcting performance issues. It is filled with lots of in-depth, real-world use-cases on solving different ElasticSearch cluster issues.

Frequently asked questions

How do I cancel my subscription?
Simply head over to the account section in settings and click on “Cancel Subscription” - it’s as simple as that. After you cancel, your membership will stay active for the remainder of the time you’ve paid for. Learn more here.
Can/how do I download books?
At the moment all of our mobile-responsive ePub books are available to download via the app. Most of our PDFs are also available to download and we're working on making the final remaining ones downloadable now. Learn more here.
What is the difference between the pricing plans?
Both plans give you full access to the library and all of Perlego’s features. The only differences are the price and subscription period: With the annual plan you’ll save around 30% compared to 12 months on the monthly plan.
What is Perlego?
We are an online textbook subscription service, where you can get access to an entire online library for less than the price of a single book per month. With over 1 million books across 1000+ topics, we’ve got you covered! Learn more here.
Do you support text-to-speech?
Look out for the read-aloud symbol on your next book to see if you can listen to it. The read-aloud tool reads text aloud for you, highlighting the text as it is being read. You can pause it, speed it up and slow it down. Learn more here.
Is Monitoring Elasticsearch an online PDF/ePUB?
Yes, you can access Monitoring Elasticsearch by Dan Noble in PDF and/or ePUB format, as well as other popular books in Computer Science & System Administration. We have over one million books available in our catalogue for you to explore.

Information

Year
2016
ISBN
9781784397807
Edition
1

Monitoring Elasticsearch


Table of Contents

Monitoring Elasticsearch
Credits
About the Author
About the Reviewers
www.PacktPub.com
eBooks, discount offers, and more
Why subscribe?
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Downloading the color images of this book
Errata
Piracy
Questions
1. Introduction to Monitoring Elasticsearch
An overview of Elasticsearch
Learning more about Elasticsearch
Data distribution, redundancy, and fault tolerance
Full-text search
Similar technologies
Apache Lucene
Solr
Ferret
Monitoring Elasticsearch
Resourcefulness and problem solving
Summary
2. Installation and the Requirements for Elasticsearch
Installing Elasticsearch
DEB/RPM installation
The yum and apt-get repositories
Ubuntu/Debian and apt-get
CentOS/RHEL and yum
Verification
Configuration files
Configuring an Elasticsearch cluster
Cluster name
Memory configuration
Open file limit
The maximum file limit
Updating max file descriptors on Ubuntu Linux
Enabling pluggable authentication modules
Verifying the open file limit
Disabling swapping
Understanding your cluster
Installing Elasticsearch-head
Installing Bigdesk
Marvel
Cluster requirements
Summary
3. Elasticsearch-head and Bigdesk
Cluster setup
Cluster configuration
Sample data
Elasticsearch-head
The Overview tab
Cluster states
Node and index actions
The Indices tab
The Browser tab
The Structured Query tab
The Any Request tab
The official website
Bigdesk
The Elasticsearch cat API
Background
Count
Health
Indices
Shards
Summary
4. Marvel Dashboard
Setting up Marvel
Upgrading Marvel
Configuring Marvel
Marvel agent configuration settings
Marvel index configuration
Understanding the Marvel dashboard
Overview dashboard
Indices dashboard
Nodes dashboard
Monitoring node failures
Summary
5. System Monitoring
Working with Kopf
Installing Kopf
The cluster page
The nodes page
The rest page
The more dropdown
Working with Logstash and Kibana
ELK
Installation
Installing Logstash
Loading NGINX logs
Installing Kibana
Working with Nagios
Installing Nagios
Command line tools for system and process management
top
tail
grep
ps
kill
free
du and df
Summary
6. Troubleshooting Performance and Reliability Issues
System configuration
The fielddata cache
Analyzing queries
Slow log
Improving query performance
High-cardinality fields
Querying smaller indices
Cold indices
The shard query cache
Script queries
Testing meticulously
System and data architecting
Hot-Warm architecture
Master nodes
Hot nodes
Warm nodes
Reducing disk size
Compression
Storing the _source and analyzed fields
Optimizing data ingestion
Bulk indexing operations
Drive configuration
Case studies
Node configuration
Query optimization
Web application performance
Summary
7. Node Failure and Post-Mortem Analysis
Diagnosing problems
OutOfMemoryError exceptions
Shard failures
Slow queries
Resolving OutOfMemoryError exceptions
Elasticsearch process crashes
Disk space
Resolving the issue
Reviewing some case studies
The ES process quits unexpectedly
Query requests slow and timing out
Summary
8. Looking Forward
Elasticsearch 5 overview
Performance and reliability
Data loss
Upgrading to Elasticsearch 5.0
When to upgrade
Monitoring Elasticsearch 5
Summary
Index

Monitoring Elasticsearch

Copyright © 2016 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the infor...

Table of contents