Learning Elasticsearch

Abhishek Andhavarapu

• Publisher: Packt Publishing
• Year: 2017
• eBook ISBN: 9781787129917
• Edition: 1
• Topic: Computer Science
• Subtopic: Data Processing
• Index: Computer Science

Store, search, and analyze your data with ease using Elasticsearch 5.xAbout This Book• Get to grips with the basics of Elasticsearch concepts and its APIs, and use them to create efficient applications• Create large-scale Elasticsearch clusters and perform analytics using aggregation• This comprehensive guide will get you up and running with Elasticsearch 5.x in no timeWho This Book Is ForIf you want to build efficient search and analytics applications using Elasticsearch, this book is for you. It will also benefit developers who have worked with Lucene or Solr before and now want to work with Elasticsearch. No previous knowledge of Elasticsearch is expected.What You Will Learn• See how to set up and configure Elasticsearch and Kibana• Know how to ingest structured and unstructured data using Elasticsearch• Understand how a search engine works and the concepts of relevance and scoring• Find out how to query Elasticsearch with a high degree of performance and scalability• Improve the user experience by using autocomplete, geolocation queries, and much more• See how to slice and dice your data using Elasticsearch aggregations.• Grasp how to use Kibana to explore and visualize your data• Know how to host on Elastic Cloud and how to use the latest X-Pack features such as Graph and AlertingIn DetailElasticsearch is a modern, fast, distributed, scalable, fault tolerant, and open source search and analytics engine. You can use Elasticsearch for small or large applications with billions of documents. It is built to scale horizontally and can handle both structured and unstructured data. Packed with easy-to- follow examples, this book will ensure you will have a firm understanding of the basics of Elasticsearch and know how to utilize its capabilities efficiently.You will install and set up Elasticsearch and Kibana, and handle documents using the Distributed Document Store. You will see how to query, search, and index your data, and perform aggregation-based analytics with ease. You will see how to use Kibana to explore and visualize your data.Further on, you will learn to handle document relationships, work with geospatial data, and much more, with this easy-to-follow guide. Finally, you will see how you can set up and scale your Elasticsearch clusters in production environments.Style and approachThis comprehensive guide will get you started with Elasticsearch 5.x, so you build a solid understanding of the basics. Every topic is explained in depth and is supplemented with practical examples to enhance your understanding.

All About Search

This chapter dives into search and helps you understand the different types of queries Elasticsearch supports. You will learn how to search, sort, and paginate on your data. Unlike SQL, the query language is based on JSON structure and is very flexible. It is very easy to combine and nest queries. You will also learn how to execute structured queries and full-text queries.

Elasticsearch is a search engine. When you run a query on Elasticsearch, each document in the result is given a relevance score. For example, you are looking for restaurant close by with decent prices, the relevance score in this case is a combination of distance and price. The results are ordered based on how relevant is each document to the query. You will learn the difference between sorting and scoring. You will learn how relevance is calculated and how to tune the relevance score.

We will discuss how to debug a search query and how it works internally. We will also go through how queries are automatically cached to improve the performance and the different types of cache.

By the end of this chapter, you will learn the following:

• Structured queries
• Full-text queries
• Sort and pagination
• Relevance
• Routing
• Caching

Different types of queries

Elasticsearch queries are executed using the Search API. Like anything else in Elasticsearch, request and response are represented in JSON.

Queries in Elasticsearch at a high level are divided as follows:

• Structured queries: Structured queries are used to query numbers, dates, statuses, and so on. These are similar to queries supported by a SQL database. For example, whether a number or date falls within a range or to find all the employees with John as the first name and so on
• Full-text search queries: Full-text search queries are used to search text fields. When you send a full-text query to Elasticsearch, it first finds all the documents that match the query, and then the documents are ranked based on how relevant each document is to the query. We will discuss relevance in detail in the Relevance section

Both structured and full-text search queries can be combined while querying. In the next section, we will describe the overall structure of request and response.

Sample data

To better explain the various concepts in this chapter, we will use the e-commerce site as an example. We will create an index with a list of products. This will be a very simple index called chapter6 with type called product. The mapping for the product type is shown here:

 #Delete existing index if any
 DELETE chapter6
 
 #Mapping
 PUT chapter6
 {
 "settings": {},
 "mappings": {
 "product": {
 "properties": {
 "product_name": {
 "type": "text",
 "analyzer": "english"
 },
 "description" : {
 "type": "text",
 "analyzer": "english"
 }
 }
 }
 }
 } 

For the product_name and description fields, the English analyzer will be used instead of the default standard analyzer. Let's index some product documents:

 #Index Documents
 PUT chapter6/product/1
 {
 "product_name": "Men's High Performance Fleece Jacket",
 "description": "Best Value. All season fleece jacket",
 "unit_price": 79.99,
 "reviews": 250,
 "release_date": "2016-08-16"
 } 
 
 PUT chapter6/product/2
 {
 "product_name": "Men's Water Resistant Jacket",
 "description": "Provides comfort during biking and hiking",
 "unit_price": 69.99,
 "reviews": 5,
 "release_date": "2017-03-02"
 } 
 
 PUT chapter6/product/3
 {
 "product_name": "Women's wool Jacket",
 "description": "Helps you stay warm in winter",
 "unit_price": 59.99,
 "reviews": 10,
 "release_date": "2016-12-15"
 } 

We will refer to the preceding three documents for different examples used in this chapter.

Querying Elasticsearch

One of most powerful features of Elasticsearch is the Query DSL (Domain specific Language) or the query language. The query language is very expressive and can be used to define filters, queries, sorting, pagination, and aggregations in the same query. To execute a search query, an HTTP request should be sent to the _search endpoint. The index and type on which the query should be executed is specified in the URL. Index and type are optional. If no index/type is specified, Elasticsearch executes the request across all the indexes in the cluster. A search query in Elasticsearch can be executed in two different ways:

• By passing the search request as query parameters.
• By passing the search request in the request body.

A simple search query using query parameters is shown here:

 GET chapter6/product/_search?q=product_name:jacket 

Simple queries can be executed using the URL request parameters. Anything other than a simple query like in the above example should be passed as the request body. The preceding query, when passed as a request body, looks like the following:

 POST chapter6/product/_search 
{
 "query": {
 "term": {
 "product_name" : "jacket"
 }
 }
 } 

The preceding query is executed on the chapter6 index and type named product. The query can also be executed on multiple indexes/types at the same time, as shown here:...

Table of contents

• Title Page
• Copyright
• Credits
• About the Author
• About the Reviewers
• www.PacktPub.com
• Customer Feedback
• Preface
• Introduction to Elasticsearch
• Setting Up Elasticsearch and Kibana
• Modeling Your Data and Document Relations
• Indexing and Updating Your Data
• Organizing Your Data and Bulk Data Ingestion
• All About Search
• More Than a Search Engine (Geofilters, Autocomplete, and More)
• How to Slice and Dice Your Data Using Aggregations
• Production and Beyond
• Exploring Elastic Stack (Elastic Cloud, Security, Graph, and Alerting)