Learn Apache Mesos

A beginner's guide to scalable cluster management and deployment

Manuj Aggarwal

• Publisher: Packt Publishing
• Year: 2018
• Print ISBN: 9781789137385
• eBook ISBN: 9781789133783
• Edition: 1
• Topic: Computer Science
• Subtopic: Cloud Computing
• Index: Computer Science

Scale applications with high availability and optimized resource management across data centers

Key Features

• Create clusters and perform scheduling, logging, and resource administration with Mesos
• Explore practical examples of managing complex clusters at scale with real-world data
• Write native Mesos frameworks with Python

Book Description

Apache Mesos is an open source cluster manager that provides efficient resource isolation and sharing across distributed applications or frameworks. This book will help you build a strong foundation of Mesos' capabilities along with practical examples to support the concepts explained throughout the book.

Learn Apache Mesos dives straight into how Mesos works. You will be introduced to the distributed system and its challenges and then learn how you can use Mesos and its framework to solve data problems. You will also gain a full understanding of Mesos' internal mechanisms and get equipped to use Mesos and develop applications. Furthermore, this book lets you explore all the steps required to create highly available clusters and build your own Mesos frameworks. You will also cover application deployment and monitoring.

By the end of this book, you will have learned how to use Mesos to make full use of machines and how to simplify data center maintenance.

What you will learn

• Deploy and monitor a Mesos cluster
• Set up servers on AWS to deploy Mesos components
• Explore Mesos resource scheduling and the allocation module
• Deploy Docker-based services and applications using Mesos Marathon
• Configure and use SSL to protect crucial endpoints of your Mesos cluster
• Debug and troubleshoot services and workloads on a Mesos cluster

Who this book is for

This book is for DevOps and data engineers and administrators who work with large data clusters. You'll also find this book useful if you have experience working with virtualization, databases, and platforms such as Hadoop and Spark. Some experience in database administration and design will help you get the most out of this book.

Apache Mesos Administration

In the last chapter, we learned how to set up a Mesos cluster in a multi-node environment. In this chapter, we will learn about different Mesos Administrative work.

In this chapter, we will learn about the following topics:

• Scheduling and allocating resources
• Understanding fault tolerance and high availability
• Configuring ZooKeeper connection information for Mesos
• ZooKeeper configuration setup on master servers
• Mesos configuration on master servers
• Configuring marathon services
• Allowing communication between subnets in AWS
• Allowing inbound access to Mesos and Marathon consoles in AWS
• Testing high availability for Mesos and Marathon services
• Starting mesos-slave and registering in Mesos Cluster

Scheduling and allocating resources

In section, we will understand

Understanding resource scheduling

Mesos has two levels of resource scheduling, which are as follows:

• In the first level, the Mesos master process gets the details of the free resources available on each node from the slave process. From there the master process gets the details of the free resources, such as CPU, memory, and so on, that are available on each slave process. Then, it groups them and offers them with a different framework, such as Marathon, Cassandra, or any of the other available frameworks available to the Mesos cluster.
• In the second level, frameworks registered as a client with the master accept or reject the offer, depending upon the requirements.
• If the offer is accepted, the framework sends the details regarding tasks and the number of each task required to the Mesos master, and then the master transfers the request to the Mesos slave to launch those tasks on nodes, and assigns required resources to the task. So, if the offer is accepted, the framework sends the details regarding tasks, such as the application, or any Docker container which it wants to run on the Mesos slave. The Mesos master gets the details from the framework, and in turn, Mesos transfers these requests to the Mesos slave to launch those tasks on a node, and assigns the required resources to the task.
• For example, the framework accepts the offer of 2 CPUs and 4 GB RAM. The Mesos master understands that offer, and looks to the Mesos slave for where the resources may be available, and allocates that application to the node wherever the resources are available. Then, the Mesos slave launches the task on nodes, and assigns the required resources to the task.
• After assigning the task, the slave manages the execution of tasks, and when the tasks are completed, the resources are freed and handed back to the Mesos master for further assignments.

Understanding resource allocation

Let's understand the resource location as follows:

• The Mesos master uses the resource allocation module to determine the type and quantity of resource offers to be made to frameworks.
• Resource allocation modules are responsible for providing shared resources in a fair manner to competing frameworks.
• Mesos has the dominant resource fairness (DRF) algorithm as its default resource allocation policy, which is far more suitable for most environments.

• So, let's consider a case where the total resources available are 8 CPUs and 10 GB of memory. User 1 runs tasks that require 1 CPU and 3 GB of memory, and user 2 runs tasks that require 3 CPUs and 1 GB of memory. Let's understand the following concepts:
• Dominant resource refers to the resources, which are CPUs and memory, that are most required by the user. In this case, user 1 runs tasks that have higher memory requirements of 3 GB per task, so the dominant resource for user 1 is memory. On the other hand, user 2 runs computation-heavy tasks using 3 CPUs per task, and hence the CPU is its dominant resource.
• In dominant share, the master calculates the dominant share and how it will distribute the resources by referring to the fraction of the dominant resource that the user is allocated. Referring to our example, user 1's dominant share becomes 30 percent, which is 3/10, whereas user 2's dominant share is 37.5 percent, which is 3/8. This is how Mesos manages resource allocation to all the tasks given by the framework.

Let's take a look at the following screenshot:

Framework 1 schedules job 1 and job 2 and the Mesos master, which has the allocation module and slave 1 and slave 2 nodes, which runs our task. The task is scheduled by the framework and managed by our Mesos master. As soon as the slave has the process of Mesos slave, the process starts. It registers the available CPU and memory to the Mesos master, which is 4 CPUs and 4 GB RAM. Then, the Mesos master offers that CPU and memory to the framework. Now, the Mesos master knows that slave 1 has 4 CPUs and 4 GB RAM. If any job requires CPU and memory, the Mesos master analyzes how much CPU and memory is required by job 1.

In our case, if we look at task 1, we can see that it asked for 2 CPUs and 1 GB RAM. The Mesos master runs job 1 on slave 1, and then job 1 is executed on slave 1. If job 2 needs to be scheduled, which requires 1 CPU and 2 GB RAM, it again goes to slave1, because it has 2 CPUs and 3 GB RAM remaining, and task 2 requires 1 CPU and 2 GB of memory. So, it again schedules that task to slave 1. This is how the Mesos master allocates and schedules the resources.

Now, in this case, slave 1 has had its resources fully allocated and executes the task. If any new task needs to be run, the Mesos master checks if it has any available resources from any other slave, and as soon as any framework requests any job to be run, the Mesos master runs that job on the node where the resources are available.

Let's understand this by visiting our Mesos cluster UI:

We can observe that we don't have the necessary resources here, as all the slave nodes are in a shutdown state. We will see how our two-level scheduling works here. First, we will start the slave server and see how the slave informs the master about the resources. So, let's go ahead and start one of the slave servers.

Start slave1 and check its status as follows:

As you can see, it hasn't started. To check this, run the following command:

Then, you will see the slave logs:

Check the error log—it's giving a Docker error.

First let's start Docker:

We can see that, as soon as we start Docker, our slave will go into a running state. Let's quickly go to our console and validate this:

In the console, the activated agent is 1, so it has a reported total of 1 CPU and 495 MB of memory, and 4 GB disk space. If you go to Agents, you will see that mesos-slave1 has been registered. Now, start the other two agents as well.

After starting Docker and the s...

Table of contents

• Title Page
• Copyright and Credits
• Packt Upsell
• Contributors
• Preface
• Deploying Apache Mesos on AWS
• Setting up Mesos Single-Cluster Nodes
• Installation of Mesosphere
• Apache Mesos Administration
• Deploying Services on Mesos Cluster
• Persistent Volumes
• Securing Mesos
• Managing Resources in Mesos
• Another Book You May Enjoy