Continuous Delivery with Docker and Jenkins

Create secure applications by building complete CI/CD pipelines, 3rd Edition

Rafal Leszko

• Publisher: Packt Publishing
• Year: 2022
• ISBN:
  9781803245300
• Edition: 3
• Topic: Computer Science
• Subtopic: System Administration
• Index: Computer Science

Create a complete continuous delivery process using modern DevOps tools such as Docker, Jenkins, Kubernetes, Ansible, Terraform, and many moreKey Features• Build reliable and secure applications using Docker containers• Create a highly available environment to scale Jenkins and your services using Kubernetes• Automate your release process end-to-endBook DescriptionThis updated third edition of Continuous Delivery with Docker and Jenkins will explain the advantages of combining Jenkins and Docker to improve the continuous integration and delivery process of app development. You'll start by setting up a Docker server and configuring Jenkins on it. Next, you'll discover steps for building applications and microservices on Dockerfiles and integrating them with Jenkins using continuous delivery processes such as continuous integration, automated acceptance testing, configuration management, and Infrastructure as Code. Moving ahead, you'll learn how to ensure quick application deployment with Docker containers, along with scaling Jenkins using Kubernetes. Later, you'll explore how to deploy applications using Docker images and test them with Jenkins. Toward the concluding chapters, the book will focus on missing parts of the CD pipeline, such as the environments and infrastructure, application versioning, and non-functional testing. By the end of this continuous integration and continuous delivery book, you'll have gained the skills you need to enhance the DevOps workflow by integrating the functionalities of Docker and Jenkins.What you will learn• Grasp Docker fundamentals and dockerize applications for the CD process• Understand how to use Jenkins on-premises and in the cloud• Scale a pool of Docker servers using Kubernetes• Write acceptance tests using Cucumber• Run tests in the Docker ecosystem using Jenkins• Provision your servers and infrastructure using Ansible and Terraform• Publish a built Docker image to a Docker registry• Deploy cycles of Jenkins pipelines using community best practicesWho this book is forThe book is for DevOps engineers, system administrators, Docker professionals, or anyone who wants to explore the power of working with Docker and Jenkins together. No prior knowledge of DevOps is required to get started.

Section 1 – Setting Up the Environment

In this section, we will be introduced to Docker, and we will cover concepts such as continuous delivery and its benefits, as well as containerization. Furthermore, we will also be introduced to the Jenkins tool, and the architecture and procedures required to install master/slave instances on a Docker server, without Docker, and using cloud environments.

The following chapters are covered in this section:

• Chapter 1, Introducing Continuous Delivery
• Chapter 2, Introducing Docker
• Chapter 3, Configuring Jenkins

Chapter 1: Introducing Continuous Delivery

A common problem that's faced by most developers is how to release the implemented code quickly and safely. The delivery process that's traditionally used is a source of pitfalls and usually leads to the disappointment of both developers and clients. This chapter will present the idea of the continuous delivery (CD) approach and provide the context for the rest of this book.

In this chapter, we will cover the following topics:

• Understanding CD
• The automated deployment pipeline
• Prerequisites to CD
• Combining CD and microservices
• Building the CD process

Understanding CD

The most accurate definition of CD is stated by Jez Humble and reads as follows:

"Continuous delivery is the ability to get changes of all types – including new features, configuration changes, bug fixes, and experiments – into production, or into the hands of users, safely and quickly, in a sustainable way."

This definition covers the key points.

To understand this better, let's imagine a scenario. You are responsible for a product – let's say, an email client application. Users come to you with a new requirement: they want to sort emails by size. You decide that the development will take around 1 week. When can the user expect to use the feature? Usually, after the development is done, you hand over the completed feature to the Quality Assurance (QA) team and then to the operations team, which takes additional time, ranging from days to months.

Therefore, even though the development only took 1 week, the user receives it in a couple of months! The CD approach addresses this issue by automating manual tasks so that the user can receive a new feature as soon as it's implemented.

To help you understand what to automate and how, we'll start by describing the delivery process that is currently used for most software systems.

The traditional delivery process

The traditional delivery process, as its name suggests, has been in place for many years and is implemented in most IT companies. Let's define how it works and comment on its shortcomings.

Introducing the traditional delivery process

Every delivery process begins with the requirements that have been defined by a customer and ends with the product being released to production. There are differences between these two stages. Traditionally, this process looks as follows:

Figure 1.1 – Release cycle diagram

The release cycle starts with the requirements provided by the Product Owner, who represents the Customer (stakeholders). Then, there are three phases, during which the work is passed between different teams:

• Development: The developers (sometimes together with business analysts) work on the product. They often use agile techniques (Scrum or Kanban) to increase the development velocity and improve communication with the client. Demo sessions are organized to obtain a customer's quick feedback. All good development techniques (such as test-driven development (TDD) or extreme programming practices) are welcome. Once the implementation is complete, the code is passed to the QA team.
• Quality Assurance: This phase is usually called User Acceptance Testing (UAT) and it requires the code to be frozen on the trunk code base so that no new development will break the tests. The QA team performs a suite of integration testing, acceptance testing, and non-functional analysis (performance, recovery, security, and so on). Any bug that is detected goes back to the development team, so the developers usually have their hands full. After the UAT phase is completed, the QA team approves the features that have been planned for the next release.
• Operations: The final phase, and usually the shortest one, involves passing the code to the operations team so that they can perform the release and monitor the production environment. If anything goes wrong, they contact the developers so that they can help with the production system.

The length of the release cycle depends on the system and the organization, but it usually ranges from 1 week to a few months. The longest I've heard about was 1 year. The longest I worked on one was quarterly-based, and each part was as follows:

• Development: 1.5 months
• UAT: 1 month and 3 weeks
• Release (and strict production monitoring): 1 week

The traditional delivery process is widely used in the IT industry, so this is probably not the first time you've read about such an approach. Nevertheless, it has several drawbacks. Let's look at them explicitly to understand why we need to strive for something better.

Shortcomings of the traditional delivery process

The most significant shortcomings of the traditional delivery process are as follows:

• Slow delivery: The customer receives the product long after the requirements were specified. This results in unsatisfactory time to market and delays customer feedback.
• Long feedback cycle: The feedback cycle is not only related to customers but developers. Imagine that you accidentally created a bug, and you learn about it during the UAT phase. How long does it take to fix something you worked on 2 months ago? Even dealing with minor bugs can take weeks.
• Lack of automation: Rare releases do not encourage automation, which leads to unpredictable releases.
• Risky hotfixes: Hotfixes cannot usually wait for the full UAT phase, so they tend to be tested differently (the UAT phase is shortened) or not tested at all.
• Stress: Unpredictable releases are stressful for the operations team. What's more, the release cycle is usually tightly scheduled, which imposes additional stress on developers and testers.
• Poor communication: Work that's passed from one team to another represents the waterfall approach, in which people start to care only about their part, rather than the complete product. If anything goes wrong, that usually leads to the blame game instead of cooperation.
• Shared responsibility: No team takes responsibility for the product from A to Z:
  • For developers: Done means that the requirements have been implemented.
  • For testers: Done means that the code has been tested.
  • For operations: Done means that the code has been released.
• Lower job satisfaction: Each phase is interesting for a different team, but other teams need to support the process. For example, the development phase is interesting for developers but, during the other two phases, they still need to fix bugs and support the release, which is usually not interesting for them at a...