Redis 4.x Cookbook

Pengcheng Huang, Zuofei Wang

• Publisher: Packt Publishing
• Year: 2018
• ISBN:
  9781783988174
• Edition: 1
• Topic: Computer Science
• Subtopic: Databases
• Index: Computer Science

Leverage the power of Redis 4.x to develop, optimize and administer your Redis solutions with easeAbout This Book• Build, deploy and administer high performance and scalable applications in Redis• Covers a range of important tasks - including development and administration of Redis• A practical guide that takes your understanding of Redis to the next levelWho This Book Is ForThis book is for database administrators, developers and architects who want to tackle the common and not so common problems associated with the different development and administration-related tasks in Redis. A fundamental understanding of Redis is expected to get the best out of this book.What You Will Learn• Install and configure your Redis instance• Explore various data types and commands in Redis• Build client-side applications as well as a Big Data framework with Redis • Manage data replication and persistence in Redis• Implement high availability and data sharding in Redis• Extend Redis with Redis Module• Benchmark, debug, fine-tune and troubleshoot various issues in RedisIn DetailRedis is considered the world's most popular key-value store database. Its versatility and the wide variety of use cases it enables have made it a popular choice of database for many enterprises. Based on the latest version of Redis, this book provides both step-by-step recipes and relevant the background information required to utilize its features to the fullest. It covers everything from a basic understanding of Redis data types to advanced aspects of Redis high availability, clustering, administration, and troubleshooting. This book will be your great companion to master all aspects of Redis.The book starts off by installing and configuring Redis for you to get started with ease. Moving on, all the data types and features of Redis are introduced in detail. Next, you will learn how to develop applications with Redis in Java, Python, and the Spring Boot web framework. You will also learn replication tasks, which will help you to troubleshoot replication issues. Furthermore, you will learn the steps that need to be undertaken to ensure high availability on your cluster and during production deployment. Toward the end of the book, you will learn the topmost tasks that will help you to troubleshoot your ecosystem efficiently, along with extending Redis by using different modules.Style and approachThis book is a rich collection of recipes that will come in handy when you are working with Redis. It addresses your common and not-so-common pain points, so this is a book of Redis that you must have on the shelf.

Setting Up High Availability and Cluster

In this chapter, we will cover the following recipes:

• Setting up Sentinel
• Testing Sentinel
• Administrating Sentinel
• Setting up Redis Cluster
• Testing Redis Cluster
• Administrating Redis Cluster

Introduction

For the requirements of a production environment, one single Redis instance is far from enough to provide a stable and efficient key-value data service with data redundancy and high availability (HA). Using the replication and persistence of Redis may solve the problem of data redundancy. However, without human intervention, the whole Redis service is not able to be recovered when the master instance is down. While various kinds of solution have been worked out for the HA of Redis, Redis Sentinel, natively supported in Redis since version 2.6, is the most widely used HA architecture. By taking advantage of Sentinel, you can easily build a fault-tolerant Redis service.

As the amounts of data stored in Redis grow rapidly, the processing power and memory capacity of a Redis instance with a large dataset (usually above 16 G) may become a bottleneck for the application. And there are more and more latency or other issues when performing persistence or replication, as the size of datasets in Redis grows. For this situation, horizontal scalability, or scaling by adding more nodes to a Redis service, is an immediate need. Redis Cluster, supported since version 3.0, is presented for this kind of problem. Redis Cluster is an out-of-the-box solution for partitioning the dataset into multiple Redis master-slave instances.

In this chapter, we will follow the pattern of Setting up, Testing, and Administrating to discuss separately how to construct a production-ready Redis service using Sentinel and Redis Cluster.

Lastly, it's worth mentioning that, as a convention, the word Cluster with the first letter capitalized refers specifically to the Redis Cluster technique. You may find various kinds of third-party systems to implement the clustering feature of Redis before version 3.0 of Redis was released. So, don't confuse the Redis Cluster we will talk about in this chapter with other data clustering systems using Redis.

Setting up Sentinel

Redis Sentinel, as the name implies, acts as a guard for the Redis master and slave instances. One Sentinel is obviously not enough to guarantee high availability because a single Sentinel itself is also subject to failure. Because the master failover decision is based on a quorum system, at least three Sentinel processes are required as a robust distributed system that keeps monitoring the status of the Redis master data server. If multiple Sentinel processes detect that the master is down, one of the Sentinel process will be elected to promote a slave to replace the old master. With proper configuration, the entire process is automatic without any human intervention. In this recipe, we will demonstrate how to set up a simple environment of one master and two slaves, which is monitored by three Sentinels.

Getting ready…

You need to set up one Redis master server and two slave servers. You can refer to the Setting up Redis replication section in Chapter 6, Persistence for instructions. In this example, we will deploy Redis Server and Sentinel instances on three different hosts, the role, IP address, and port, shown in the following table:

Role	IP address	Port
Master	192.168.0.31	6379
Slave-1	192.168.0.32	6379
Slave-2	192.168.0.33	6379
Sentinel-1	192.168.0.31	26379
Sentinal-2	192.168.0.32	26379
Sentinel-3	192.168.0.33	26379

The overall architecture can be seen in the following image:

The three hosts must be able to communicate with each other. You need to set the binding IP correctly in the Redis configuration file, in order to let other hosts talk to the Redis instance. The default binding IP is 127.0.0.1, which only enables access from the localhost. You can append the IP address as follows:

bind 127.0.0.1 192.168.0.31 

Make sure all Redis data servers are up and running.

How to do it...

The steps for setting up Sentinel are as follows:

1. Prepare a configuration file, sentinel.conf, on each host. You can get a sample copy from the source code. Make sure the configuration file is writable by the user who runs the Sentinel process:

port 26379 dir /tmp sentinel monitor mymaster 192.168.0.31 6379 2 sentinel down-after-milliseconds mymaster 30000 sentinel parallel-syncs mymaster 1 sentinel failover-timeout mymaster 180000 

2. Start Sentinel processes on three hosts:

[email protected]:~$bin/redis-server conf/sentinel.conf --sentinel [email protected]:~$bin/redis-server conf/sentinel.conf --sentinel [email protected]:~$bin/redis-server conf/sentinel.conf --sentinel 

3. Examine the logs on Sentinel-1:

21758:X 29 Oct 22:31:51.001 # Sentinel ID is 3ef95f7fd6420bfe22e38bfded1399382a63ce5b 21758:X 29 Oct 22:31:51.001 # +monitor master mymaster 192.168.0.31 6379 quorum 2 21758:X 29 Oct 22:31:51.001 * +slave slave 192.168.0.32:6379 192.168.0.32 6379 @ mymaster 192.168.0.31 6379 21758:X 29 Oct 22:31:51.003 * +slave slave 192.168.0.33:6379 192.168.0.33 6379 @ mymaster 192.168.0.31 6379 21758:X 29 Oct 22:31:52.021 * +sentinel sentinel d24979c27871eafa62e797d1c8e51acc99bbda72 192.168.0.32 26379 @ mymaster 192.168.0.31 6379 21758:X 29 Oct 22:32:17.241 * +sentinel sentinel a276b044b26100570bb1a4d83d5b3f9d66729f64 192.168.0.33 26379 @ mymaster 192.168.0.31 6379 

4. Connect to Sentinel-1 using redis-cli and execute the INFO SENTINEL command. Please don't forget to specify the port, 26379:

[email protected]:~$ bin/redis-cli -p 26379 127.0.0.1:26379> INFO SENTINEL # Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 master0:name=mymaster,status=ok,address=192.168.0.31:6379,slaves=2,sentinels=3 

5. Examine the content of sentinel.conf on Sentinel-1:

[email protected]:~$ cat conf/sentinel.c...