![]()
![]()
Table of Contents
Mastering Mesos
Credits
About the Authors
About the Reviewer
www.PacktPub.com
eBooks, discount offers, and more
Why subscribe?
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Errata
Piracy
Questions
1. Introducing Mesos
Introduction to the datacenter OS and architecture of Mesos
The architecture of Mesos
Introduction to frameworks
Frameworks built on Mesos
Long-running services
Big data processing
Batch scheduling
Data storage
The attributes and resources of Mesos
Attributes
Resources
Examples
Two-level scheduling
Resource allocation
Max-min fair share algorithm
Resource isolation
Monitoring in Mesos
Monitoring provided by Mesos
Types of metrics
The Mesos API
Messages
API details
Executor API
The Executor Driver API
The Scheduler API
The Scheduler Driver API
Mesos in production
Case study on HubSpot
The cluster environment
Benefits
Challenges
Looking ahead
Summary
2. Mesos Internals
Scaling and efficiency
Resource allocation
The Dominant Resource Fairness algorithm (DRF)
Weighted DRF
Configuring resource offers on Mesos
Reservation
Static reservation
Role definition
Framework assignment
Role resource policy setting
Dynamic reservation
Offer::Operation::Reserve
Offer::Operation::Unreserve
/reserve
/unreserve
Oversubscription
Revocable resource offers
Registering with the revocable resources capability
An example offer with a mix of revocable and standard resources
Resource estimator
The QoS controller
Configuring oversubscription
Extendibility
Mesos modules
Module invocation
Building a module
Hooks
The currently supported modules
The allocator module
Implementing a custom allocator module
High availability and fault tolerance
Mastering high availability
Framework scheduler fault tolerance
Slave fault tolerance
Executor/task
Slave recovery
Enabling slave checkpointing
Enabling framework checkpointing
Reconciliation
Task reconciliation
Offer reconciliation
Persistent Volumes
Offer::Operation::Create
Offer::Operation::Destroy
Summary
3. Getting Started with Mesos
Virtual Machine (VM) instances
Setting up a multi-node Mesos cluster on Amazon Web Services (AWS)
Instance types
Launching instances
Installing Mesos
Downloading Mesos
Building Mesos
Using mesos-ec2 script to launch many machines at once
Setting up a multi-node Mesos cluster on Google Compute Engine (GCE)
Introduction to instance types
Launching machines
Set up a Google Cloud Platform project
Create the network and firewall rules
Create the instances
Installing Mesos
Downloading Mesos
Building Mesos
Setting up a multi-node Mesos cluster on Microsoft Azure
Introduction to instance types
Launching machines
Create a cloud service
Create the instances
Configuring the network
Installing Mesos
Downloading Mesos
Building Mesos
Starting mesos-master
Start mesos-slaves
Mesos commands
Testing the installation
Setting up a multi-node Mesos cluster on your private datacenter
Installing Mesos
Preparing the environment
Downloading Mesos
Building Mesos
Starting mesos-master
Starting mesos-slaves
Automating the process when you have many machines
Debugging and troubleshooting
Handling missing library dependencies
Issues with directory permissions
Missing Mesos library (libmesos*.so not found)
Debugging a failed framework
Understanding the Mesos directory structure
Mesos slaves are not connecting with Mesos masters
Launching multiple slave instances on the same machine
Summary
4. Service Scheduling and Management Frameworks
Using Marathon to launch and manage long-running applications on Mesos
Installing Marathon
Installing ZooKeeper to store the state
Launching Marathon in local mode
Multi-node Marathon cluster setup
Launching a test application from the UI
Scaling the application
Terminating the application
Chronos as a cluster scheduler
Installing Chronos
Scheduling a new job
Chronos plus Marathon
The Chronos REST API endpoint
Listing the running jobs
Manually starting a job
Adding a scheduled job
Deleting a job
Deleting all the tasks of a job
The Marathon REST API endpoint
Listing the running applications
Adding an application
Changing the configuration of an application
Deleting the application
Introduction to Apache Aurora
Installing Aurora
Introduction to Singularity
Installing Singularity
Creating a Singularity configuration file
Service discovery using Marathoner
Service discovery using Consul
Running Consul
Load balancing with HAProxy
Creating the bridge between HAProxy and Marathon
Bamboo - Automatically configuring HAProxy for Mesos plus Marathon
Introduction to Netflix Fenzo
Introduction to PaaSTA
A comparative analysis of different Scheduling/Management frameworks
Summary
5. Mesos Cluster Deployment
Deploying and configuring a Mesos cluster using Ansible
Installing Ansible
Installing the control machine
Creating an ansible-mesos setup
Deploying and configuring Mesos cluster using Puppet
Deploying and configuring a Mesos cluster using SaltStack
SaltStack installation
Deploying and configuring a Mesos cluster using Chef
Recipes
Configuring mesos-master
Configuring mesos-slave
Deployin...
