Docker on Amazon Web Services

Build, deploy, and manage your container applications at scale

Justin Menga

• Publisher: Packt Publishing
• Year: 2018
• ISBN:
  9781788622721
• Edition: 1
• Topic: Computer Science
• Subtopic: Cloud Computing
• Index: Computer Science

Run Docker on AWS and build real-world, secure, and scalable container platforms on cloud

Key Features

• Configure Docker for the ECS environment
• Integrate Docker with different AWS tools
• Implement container networking and deployment at scale

Book Description

Over the last few years, Docker has been the gold standard for building and distributing container applications. Amazon Web Services (AWS) is a leader in public cloud computing, and was the first to offer a managed container platform in the form of the Elastic Container Service (ECS).

Docker on Amazon Web Services starts with the basics of containers, Docker, and AWS, before teaching you how to install Docker on your local machine and establish access to your AWS account. You'll then dig deeper into the ECS, a native container management platform provided by AWS that simplifies management and operation of your Docker clusters and applications for no additional cost. Once you have got to grips with the basics, you'll solve key operational challenges, including secrets management and auto-scaling your infrastructure and applications. You'll explore alternative strategies for deploying and running your Docker applications on AWS, including Fargate and ECS Service Discovery, Elastic Beanstalk, Docker Swarm and Elastic Kubernetes Service (EKS). In addition to this, there will be a strong focus on adopting an Infrastructure as Code (IaC) approach using AWS CloudFormation.

By the end of this book, you'll not only understand how to run Docker on AWS, but also be able to build real-world, secure, and scalable container platforms in the cloud.

What you will learn

• Build, deploy, and operate Docker applications using AWS
• Solve key operational challenges, such as secrets management
• Exploit the powerful capabilities and tight integration of other AWS services
• Design and operate Docker applications running on ECS
• Deploy Docker applications quickly, consistently, and reliably using IaC
• Manage and operate Docker clusters and applications for no additional cost

Who this book is for

Docker on Amazon Web Services is for you if you want to build, deploy, and operate applications using the power of containers, Docker, and Amazon Web Services. Basic understanding of containers and Amazon Web Services or any other cloud provider will be helpful, although no previous experience of working with these is required.

Elastic Kubernetes Service

Kubernetes is a popular open source container management platform originally developed by Google, who based Kubernetes on Google's own internal Borg (https://kubernetes.io/blog/2015/04/borg-predecessor-to-kubernetes/) container platform. Kubernetes draws on Google's extensive experience of running containers at scale, and is now supported by all of the major cloud platform providers with the release of the AWS Elastic Kubernetes Service (EKS). EKS provides a managed Kubernetes cluster to which you can deploy your container applications, without having to worry about day-to-day operational overheads and the complexities of cluster management. AWS has performed all the heavy lifting of establishing a robust and scalable platform, making it easier than ever to get up and running with Kubernetes.

In this chapter, you will be introduced to the world of Kubernetes, we will work through how we can configure Kubernetes to ensure we are able to successfully deploy and operate the sample application we have used through this book, and then establish an EKS cluster in AWS where you will deploy the application using the configuration you have developed locally. This will provide practical, real-world insights into how, as an application owner, you can deploy your container workloads to Kubernetes, and how you can quickly get up and running with EKS.

We will first learn how you can work with the platform locally, using the native support that Docker for Mac and Docker for Windows now include for Kubernetes. You can spin up a local single-node cluster straight out of the box, reducing much of the manual configuration you would typically require to get a local environment up and running. You will learn how to create the various types of resources required to run the sample application in Kubernetes, addressing key operational challenges such as providing persistent storage for your application database, secrets management, and running one-shot tasks such as database migrations.

Once you have established a working configuration to get the sample application up and running locally in Kubernetes, we will turn our attention to getting started with EKS, creating an EKS cluster, and establishing an EC2 auto scaling group where worker nodes that run your container workloads are managed. You will learn how to set up access to your cluster from your local environment and proceed to deploy the Kubernetes Dashboard, which provides a rich management user interface from which you can deploy and manage your applications. Finally, you will set up integrations with other AWS services including Elastic Block Store (EBS) and Elastic Load Balancing (ELB), and proceed to deploy the sample application to your EKS cluster.

The following topics will be covered in this chapter:

• Introduction to Kubernetes
• Kubernetes architecture
• Getting started with Kubernetes
• Installing Kubernetes using Docker Desktop
• Creating core Kubernetes resources including pods, deployments, and services
• Creating Persistent volumes
• Creating Kubernetes secrets
• Running Kubernetes jobs
• Creating an EKS cluster
• Establishing access to an EKS cluster
• Deploying applications to EKS

Technical requirements

The following are the technical requirements for this chapter:

• Administrator access to an AWS account
• A local AWS profile, configured as per the instructions in Chapter 3
• AWS CLI version 1.15.71 or higher
• Docker 18.06 or higher
• Docker Compose 1.22 or higher
• GNU Make 3.82 or higher
• This chapter assumes you have completed all of the preceding chapters in this book

The following GitHub URL contains the code samples used in this chapter: https://github.com/docker-in-aws/docker-in-aws/tree/master/ch17.

Check out the following video to see the Code in Action:
http://bit.ly/2LyGtSY

Introduction to Kubernetes

Kubernetes is an open source container management platform that was open sourced by Google in 2014, and achieved production readiness in 2015 with its 1.0 release. In the space of three years, it has established itself as the most popular container management platform, and is very popular for larger organizations that are looking to run their applications as container workloads. Kubernetes is one of the most popular open source projects (https://github.com/cncf/velocity/blob/master/docs/top30_chart_creation.md) on GitHub, and according to Redmonk, Kubernetes is used at 54% of Fortune 100 companies as of late 2017.

Key features of Kubernetes include the following:

• Platform agnostic: Kubernetes can run anywhere, from your local machine to your data centre and in cloud providers such as AWS, Azure, and Google Cloud, whom all now offer integrated managed Kubernetes offerings.
• Open source: Kubernetes' greatest strength is its community and open source nature, which has seen Kubernetes become one of the leading open source projects on the planet. Major organizations and vendors are investing significant time and resources contributing to the platform, ensuring that the entire community benefits from these ongoing enhancements.
• Pedigree: Kubernetes' roots are from Google's internal Borg platform, which has been running containers at scale since the early 2000s. Google is one of the pioneers of container technology and is without a doubt one of, if not the largest, adopters of containers – back in 2014, Google indicated that they were running 2 billion containers every week, at a time when most enterprises had only just heard about containers through a new project called Docker that was taking the tech industry by storm. This pedigree and heritage ensures many of the lessons Google has learned over its many years of running containers at scale are encapsulated in the Kubernetes platform.

• Production grade container management features: Kubernetes offers all of the container management features that you would expect to see and will come across with other competing platforms. This includes cluster management, multi-host networking, pluggable storage, health checks, service discovery and load balancing, service scaling and rolling updates, desired stage configuration, role-based access control, and secret management to name a few. All of these features are implemented in a modular building-block fashion that allows you to tune the system to meet the specific requirements of your organization, and is one of the reasons Kubernetes is now considered the gold standard for enterprise-grade container management.

Kubernetes versus Docker Swarm

In the previous chapter, I provided my own thoughts on Docker Swarm versus Kubernetes, and here I will continue, this time with more of a focus on why you would choose Kubernetes over Docker Swarm. As you work through this chapter, it should become apparent that Kubernetes has a more elaborate architecture that means there is a higher learning curve, and what I cover in this chapter only really scratches the surface of what is possible with Kubernetes. That said, once you get your head around these concepts, at least from my perspective, you should see that ultimately Kubernetes is more powerful with greater flexibility, and arguably it's probably fair to state that Kubernetes certainly feels more "enterprise-grade" than Docker Swarm, with many more knobs you can tune to tailor Kubernetes to your specific needs.

Probably the biggest advantage Kubernetes has over Docker Swarm and other competitors is its community, which is significant and means that information about almost any configuration scenario you can think of, can be readily found across the wider Kubernetes community and ecosystem. There has been a lot of momentum behind the Kubernetes movement, and this only seems to be growing as leading vendors and providers such as AWS embrace Kubernetes wi...