Elasticsearch Indexing

Huseyin Akdogan

• Publisher: Packt Publishing
• Year: 2015
• eBook ISBN: 9781783987030
• Edition: 1
• Topic: Computer Science
• Subtopic: Web Development
• Index: Computer Science

Improve search experiences with ElasticSearch's powerful indexing functionality – learn how with this practical ElasticSearch tutorial, packed with tips!About This Book• Improve user's search experience with the correct configuration• Deliver relevant search results – fast!• Save time and system resources by creating stable clustersWho This Book Is ForIf you understand the importance of a great search experience this book will show you exactly how to build one with ElasticSearch, one of the world's leading search servers.What You Will Learn• Learn how ElasticSearch efficiently stores data – and find out how it can reduce costs• Control document metadata with the correct mapping strategies and by configuring indices• Use ElasticSearch analysis and analyzers to incorporate greater intelligence and organization across your documents and data• Find out how an ElasticSearch cluster works – and learn the best way to configure it• Perform high-speed indexing with low system resource cost• Improve query relevance with appropriate mapping, suggest API, and other ElasticSearch functionalitiesIn DetailBeginning with an overview of the way ElasticSearch stores data, you'll begin to extend your knowledge to tackle indexing and mapping, and learn how to configure ElasticSearch to meet your users' needs. You'll then find out how to use analysis and analyzers for greater intelligence in how you organize and pull up search results – to guarantee that every search query is met with the relevant results! You'll explore the anatomy of an ElasticSearch cluster, and learn how to set up configurations that give you optimum availability as well as scalability. Once you've learned how these elements work, you'll find real-world solutions to help you improve indexing performance, as well as tips and guidance on safety so you can back up and restore data. Once you've learned each component outlined throughout, you will be confident that you can help to deliver an improved search experience – exactly what modern users demand and expect.Style and approachThis is a comprehensive guide to performing efficient indexing and providing relevant search results using mapping, analyzers, and other ElasticSearch functionalities.

Elasticsearch Indexing

---

Table of Contents

Elasticsearch Indexing

Credits

About the Author

About the Reviewer

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Introduction to Efficient Indexing

Getting started

Understanding the document storage strategy

The _source field

The difference between the storable and searchable field

Analysis

Summary

2. What is an Elasticsearch Index

Nature of the Elasticsearch index

Indices

Mapping

Types

Document

Denormalization

Inverted index

Summary

3. Basic Concepts of Mapping

Basic concepts and definitions

Metadata fields

_source

_all

_timestamp

_ttl

Types

Object type

Root object type

Attachment type

The relationship between mapping and relevant search results

Understanding the schema-less

Summary

4. Analysis and Analyzers

Introducing analysis

Process of analysis

Built-in analyzers

Building blocks of Analyzer

Characte filters

HTML Strip Char filter

Pattern Replace Char filter

Tokenizer

Token filters

What's text normalization?

ICU analysis plugin

ASCII Foldng Token filter

An Analyzer Pipeline

Specifying the analyzer for a field in the mapping

Creating a custom analyzer

Summary

5. Anatomy of an Elasticsearch Cluster

Basic concepts

Node

Non-data nodes

Dedicated master nodes

Client nodes

Tribe node

Shards

Replicas

Explaining the architecture of distribution

Correctly configuring the cluster

Choosing the right amount of shards and replicas

Summary

6. Improving Indexing Performance

Configuration

Memory configuration

The ES_HEAP_SIZE environment variable

Avoiding swapping

Mlockall property

Garbage collector

The structure of JVM memory

What is the problem?

Monitoring garbage collection

VisualVM

Different strategies among garbage collectors

Process of deallocating memory

Types of garbage collector

Serial garbage collector

Parallel garbage collector

Concurrent Mark Sweep garbage collector

G1 garbage collector

Tuning the garbage collection

File descriptors

Increasing FD limit on Unix systems

Optimization of mapping definition

Norms

Feature index_option of string type

Exclude unnecessary fields

Extension of the automatic index refresh time

Segments and merging policies

Choosing the right merge policy

Tiered policy

log_byte_size policy

Log_doc policy

The optimize API

Store module

Store types

Simple filesystem store

New IO filesystem store

MMap filesystem store

Hybrid filesystem store

Throttling I/O operations

Throttling type

Bulk API

Bulk sizing

Notes

Summary

7. Snapshot and Restore

Snapshot repository

Repository types

Shared filesystem repository

URL repository

Cloud repository

HDFS filesystem repository

Snapshot

Restore

Overriding index settings during restore

How does the snapshot process works?

Summary

8. Improving the User Search Experience

Correction of users' spelling mistakes

Suggesters

Using the _suggest REST endpoint

Suggest object inclusion in the query

Term suggester

Configuring the term suggester

Common suggest options

Other and additional term suggester options

The phrase suggester

Configuring the phrase suggester

The completion suggester

Mapping the configuration for the completion suggester

Indexing on completion field

Get suggestions

Improving the relevancy of search results

Boosting the query

Bool query

Synonyms

Be careful about the _all field

Summary

Index

Elasticsearch Indexing

Copyright © 2015 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: December 2015

Production reference: 1171215

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78398-702-3

www.packtpub.com

Credits

Author

Hüseyin Akdoğan

Reviewer

John M. Petrone

Commissioning Editor

Kartikey Pandey

Acquisition Editor

Shaon Basu

Content Development Editor

Anish Dhurat

Technical Editor

Pranjali Mistry

Copy Editor

Neha Vyas

Project Coordinator

Bijal Patel

Proofreader

Safis Editing

Indexer

Mariammal Chettiyar

Graphics

Disha Haria

Production Coordinator

Nilesh Mohite

Cover Work

Nilesh Mohite

About the Author

Hüseyin Akdoğan began his software adventure with the GwBasic programming language. He started learning the Visual Basic language after QuickBasic and developed many applications until 2000, after which he stepped into the world of Web with PHP. After this, he came across Java! In addition to counseling and training activities since 2005, he developed enterprise applications with JavaEE technologies. His areas of expertise are JavaServer Faces, Spring Frameworks, and big data technologies such as NoSQL and Elasticsearch. Along with these, he is also trying to specialize in other big data technologies. Hüseyin also writes articles on Java and big data technologies and works as a technical reviewer of big data books. He was a reviewer of one of the bestselling books, Mastering Elasticsearch – Second Edition.

About the Reviewer

John M. Petrone is a veteran technology leader and innovator who has over 20 years of experience in leading software development and technical operations at organizations ranging in size and scope from early-stage start-ups to public companies and large system integrators. He's passionate about the strategic ap...

Table of contents

• Elasticsearch Indexing