Continuous Delivery with Docker and Jenkins

Create secure applications by building complete CI/CD pipelines, 2nd Edition

Rafał Leszko

• Publisher: Packt Publishing
• Year: 2019
• ISBN:
  9781838553081
• Edition: 2
• Topic: Computer Science
• Subtopic: Software Development
• Index: Computer Science

Create a complete Continuous Delivery process using modern DevOps tools such as Docker, Kubernetes, Jenkins, Docker Hub, Ansible, GitHub and many more.

Key Features

• Build reliable and secure applications using Docker containers.
• Create a highly available environment to scale a Docker servers using Kubernetes
• Implement advance continuous delivery process by parallelizing the pipeline tasks

Book Description

Continuous Delivery with Docker and Jenkins, Second Edition will explain the advantages of combining Jenkins and Docker to improve the continuous integration and delivery process of an app development. It will start with setting up a Docker server and configuring Jenkins on it. It will then provide steps to build applications on Docker files and integrate them with Jenkins using continuous delivery processes such as continuous integration, automated acceptance testing, and configuration management.

Moving on, you will learn how to ensure quick application deployment with Docker containers along with scaling Jenkins using Kubernetes. Next, you will get to know how to deploy applications using Docker images and testing them with Jenkins. Towards the end, the book will touch base with missing parts of the CD pipeline, which are the environments and infrastructure, application versioning, and nonfunctional testing.

By the end of the book, you will be enhancing the DevOps workflow by integrating the functionalities of Docker and Jenkins.

What you will learn

• Get to grips with docker fundamentals and how to dockerize an application for the CD process
• Learn how to use Jenkins on the Cloud environments
• Scale a pool of Docker servers using Kubernetes
• Create multi-container applications using Docker Compose
• Write acceptance tests using Cucumber and run them in the Docker ecosystem using Jenkins
• Publish a built Docker image to a Docker Registry and deploy cycles of Jenkins pipelines using community best practices

Who this book is for

The book targets DevOps engineers, system administrators, docker professionals or any stakeholders who would like to explore the power of working with Docker and Jenkins together. No prior knowledge of DevOps is required for this book.

Section 1: Setting Up the Environment

In this section, you will be introduced to Docker,and we will cover concepts such as Continuous Delivery and its benefits, as well as containerization. Furthermore, we will also be introduced to the Jenkins tool, and the architecture and procedures required to install master/slave instances on a Docker server, without Docker, and using cloud environments.

The following chapters are covered in this section:

• Chapter 1, Introducing Continuous Delivery
• Chapter 2, Introducing Docker
• Chapter 3, Configuring Jenkins

Introducing Continuous Delivery

A common problem faced by most developers is how to release the implemented code quickly and safely. The delivery process used traditionally is a source of pitfalls and usually leads to the disappointment of both developers and clients. This chapter presents the idea of the Continuous Delivery (CD) approach and provides the context for the rest of the book.

This chapter covers the following points:

• Understanding CD
• The automated deployment pipeline
• Prerequisites to CD
• Building the CD process
• Creating a complete CD system

Understanding CD

The most accurate definition of the CD is stated by Jez Humble and reads as follows:

"Continuous Delivery is the ability to get changes of all types—including new features, configuration changes, bug fixes, and experiments—into production, or into the hands of users, safely and quickly, in a sustainable way."

This definition covers the key points.

To understand it better, let's imagine a scenario. You are responsible for a product, let's say, the email client application. Users come to you with a new requirement: they want to sort emails by size. You decide that the development will take around one week. When can the user expect to use the feature? Usually, after the development is done, you hand over the completed feature first to the QA team and then to the operations team, which takes additional time, ranging from days to months.

Therefore, even though the development took only one week, the user receives it in a couple of months! The CD approach addresses that issue by automating manual tasks so that the user could receive a new feature as soon as it's implemented.

To help you to understand what to automate and how, let's start by describing the delivery process that is currently used for most software systems.

The traditional delivery process

The traditional delivery process, as the name suggests, has been in place for many years and is implemented in most IT companies. Let's define how it works and comment on its shortcomings.

Introducing the traditional delivery process

Any delivery process begins with the requirements defined by a customer and ends up with release on production. The differences are in-between. Traditionally, it looks as presented in the following release cycle diagram:

The release cycle starts with the requirements provided by the Product Owner, who represents the Customer (stakeholders). Then there are three phases, during which the work is passed between different teams:

• Development: The developers (sometimes together with business analysts) work on the product. They often use Agile techniques (Scrum or Kanban) to increase the development velocity and to improve communication with the client. Demo sessions are organized to obtain a customer's quick feedback. All good development techniques (such as test-driven development or extreme programming practices) are welcome. Once implementation is complete, the code is passed to the QA team.
• Quality Assurance: This phase is usually called User Acceptance Testing (UAT) and it requires a code freeze on the trunk code base, so that no new development would break the tests. The QA team performs a suite of Integration Testing, Acceptance Testing, and Non-functional analysis (performance, recovery, security, and so on). Any bug that is detected goes back to the development team, so developers usually have their hands full. After the UAT phase is completed, the QA team approves the features that are planned for the next release.
• Operations: The final phase, usually the shortest one, means passing the code to the operations team, so that they can perform the release and monitor production. If anything goes wrong, they contact developers to help with the production system.

The length of the release cycle depends on the system and the organization, but it usually ranges from a week to a few months. The longest I've heard about was one year. The longest I worked with was quarterly-based, and each part took as follows: development—1.5 months, UAT—1 month and 3 weeks, release (and strict production monitoring)—1 week.

The traditional delivery process is widely used in the IT industry and it's probably not the first time you've read about such an approach. Nevertheless, it has a number of drawbacks. Let's look at them explicitly to understand why we need to strive for something better.

Shortcomings of the traditional delivery process

The most significant shortcomings of the traditional delivery process include the following:

• Slow delivery: The customer receives the product long after the requirements were specified. This results in unsatisfactory time to market and delays customer feedback.
• Long feedback cycle: The feedback cycle is not only related to customers, but also to developers. Imagine that you accidentally created a bug and you learn about it during the UAT phase. How long does it take to fix something you worked on two months ago? Even dealing with minor bugs can take weeks.
• Lack of automation: Rare releases don't encourage automation, which leads to unpredictable releases.
• Risky hotfixes: Hotfixes can't usually wait for the full UAT phase, so they tend to be tested differently (the UAT phase is shortened) or not tested at all.
• Stress: Unpredictable releases are stressful for the operations team. What's more, the release cycle is usually tightly scheduled, which imposes an additional stress on developers and testers.
• Poor communication: Work passed from one team to another represents the waterfall approach, in which people start to care only about their part, rather than the complete product. In case anything goes wrong, that usually leads to the blame game instead of cooperation.
• Shared responsibility: No team takes responsibility for the product from A to Z:
  • For developers: done means that requirements are implemented
  • For testers: done means that the code is tested
  • For operations: done means that the code is released

• Lower job satisfaction: Each phase is interesting for a different team, but other teams need to support the process. For example, the development phase is interesting for developers but, during the other two phases, they still need to fix bugs and support the release, which usually is not interesting for them at all.

These drawbacks represent just a tip of the iceberg of the challenges related to the traditional delivery process. You may already feel that there must be a better way to develop the software and this better way is, obviously, the CD approach.

Benefits of CD

How long would it take your organization to deploy a change that involves just a single line of code? Do you do this on a repeatable, reliable basis? These are the famous questions from Mary and Tom Poppendieck (authors of Implementing Lean Software Development), which have been quoted many times by Jez Humble and others. Actually, the answer to these questions is the only valid measurement of the health of your delivery process.

To be able to deliver continuously, and not spend a fortune on the army of operations, teams working 24/7, we need automation. That is why, in short, CD is all about changing each phase of the traditional delivery process into a sequence of scripts, called the automated deployment pipeline, or the CD pipeline. Then, if no manual steps are required, we can run the process after every code change and, therefore, deliver the product continuously to users.

CD lets us get rid of the tedious release cycle and, therefore, brings the following benefits:

• Fast delivery: Time to market is significantly reduced as customers can use the product as soon as development is completed. Remember that the software delivers no revenue until it is in the hands of its users.
• Fast feedback cycle: Imagine you created a bug in the code, which goes into production the same day. How much time does it take to fix something you worked on the same day? Probably not much. This, together with the quick rollback strategy, is the best way to keep...