Jenkins Administrator's Guide

Calvin Sangbin Park, Lalit Adithya, Samuel Gleske, Tracy Miranda

• Publisher: Packt Publishing
• Year: 2021
• eBook ISBN: 9781838828295
• Edition: 1
• Topic: Computer Science
• Subtopic: Business Intelligence
• Index: Computer Science

Build and manage a production Jenkins instance, complete with CI/CD pipelines using GitHub and Docker Hub, Jenkins Configuration as Code, Shared Libraries, Script Security, and optimization guidesKey Features• Set up production-grade Jenkins and CI/CD pipelines with GitHub and Docker Hub integrations• Manage, protect, and upgrade a production Jenkins instance regardless of its size and the number of users• Scale a Jenkins instance using advanced optimization tips, tricks, and best practicesBook DescriptionJenkins is a renowned name among build and release CI/CD DevOps engineers because of its usefulness in automating builds, releases, and even operations. Despite its capabilities and popularity, it's not easy to scale Jenkins in a production environment. Jenkins Administrator's Guide will not only teach you how to set up a production-grade Jenkins instance from scratch, but also cover management and scaling strategies.This book will guide you through the steps for setting up a Jenkins instance on AWS and inside a corporate firewall, while discussing design choices and configuration options, such as TLS termination points and security policies. You'll create CI/CD pipelines that are triggered through GitHub pull request events, and also understand the various Jenkinsfile syntax types to help you develop a build and release process unique to your requirements. For readers who are new to Amazon Web Services, the book has a dedicated chapter on AWS with screenshots. You'll also get to grips with Jenkins Configuration as Code, disaster recovery, upgrading plans, removing bottlenecks, and more to help you manage and scale your Jenkins instance.By the end of this book, you'll not only have a production-grade Jenkins instance with CI/CD pipelines in place, but also knowledge of best practices by industry experts.What you will learn• Set up a production-grade Jenkins instance on AWS and on-premises• Create continuous integration and continuous delivery (CI/CD) pipelines triggered by GitHub pull request events• Use Jenkins Configuration as Code to codify a Jenkins setup• Backup and restore configurations and plan for disaster recovery• Plan, communicate, execute, and roll back upgrade scenarios• Identify and remove common bottlenecks in scaling Jenkins• Use Shared Libraries to develop helper functions and create new DSLsWho this book is forThis book is for both new Jenkins administrators and advanced users who want to optimize and scale Jenkins. Jenkins beginners can follow the step-by-step directions, while advanced readers can join in-depth discussions on Script Security, removing bottlenecks, and other interesting topics. Build and release CI/CD DevOps engineers of all levels will also find new and useful information to help them run a production-grade Jenkins instance following industry best practices.

1

Jenkins Infrastructure with TLS/SSL and Reverse Proxy

In this chapter, we will learn about the foundational components of Jenkins: the controller, agents, cloud, domain name, TLS/SSL certificates, and reverse proxy. First, we will learn where each component fits into the architecture, and then prepare the VMs and TLS/SSL certificates. Finally, we will learn the importance of choosing the right storage medium for the Jenkins controller and discuss the pros and cons of some of the popular storage options. By the end of this chapter, we will understand the Jenkins architecture and have the necessary components ready so that we can put them together in the next chapter.

In this chapter, we're going to cover the following main topics:

• Why Jenkins?
• Searching for answers online with Jenkins keywords
• Understanding the Jenkins architecture
• AWS: FAQs, routing rules, EC2 instances, and EIPs
• Installing Docker on our VMs
• Acquiring domain names and TLS/SSL certificates
• Storage concerns (very important!)

Technical requirements

You need a domain name (for example, jenkins.example.com) for your Jenkins instance and one or both of the following:

• An AWS account with permission to create three EC2 instances, create an Application ELB, create certificates via AWS Certificate Manager, create AWS access keys via IAM, and modify domain records in Route 53.
• Three VMs running Ubuntu 20.04, access to domain records for your domain, and optionally a company public key infrastructure (PKI).

The files for this chapter are available in this book’s GitHub repository at https://github.com/PacktPublishing/Practical-Jenkins-2.x/blob/main/ch1.

Why Jenkins?

A Continuous Integration (CI) build runs when a pull request (PR) is created or updated so that a developer can build and test the proposed change before merging the change. In the 2020s, a CI system is a normal fact of life for any software development, and it is difficult to imagine developing a software product without it.

But finding a good CI solution remains difficult, particularly when the validation process is more complex than a simple use case of building a Go binary and creating a Docker image with it. For example, if we need to build a phone firmware and need to flash a physical phone in one of the three test phones as a part of the validation, we would need a CI system that can handle physical connections to the hardware, as well as manage resource locks to make sure that the validation does not disrupt other validations that are already in progress.

The use cases are even more complex during the Continuous Delivery or Continuous Deployment (CD) process, where the end product of a build is either stored in an archive (such as Docker registry or Artifactory) or deployed to a production system. The CD builds' more frequent interactions with external systems require credential management and environment preparation for a deployment. For example, it's not uncommon for a production system to be in an isolated network inaccessible from the corporate network, and a jumpbox session must be prepared before an update can be deployed. It's also common that the deployments are handled in a different set of pipelines that are not tied to the code changes in the Git repository, which means that we would need a CD solution made up of free-standing pipelines that are detached from any PR activities.

Jenkins, known for its supreme flexibility, can handle such complex use cases easily. Managing a physical hardware connection, handling resource locks, managing credentials, and handling session data across multiple stages is all built into Jenkins or can be made available easily through one of 1,500+ plugins1.

In addition to the rich feature set, Jenkins supports real programming languages such as Java and Groovy to specify the build steps. This allows us to write real code to express our build logic using variables, conditionals, loops, exceptions, and sometimes even classes and methods, rather than being bound by a domain-specific language (DSL) that is often limiting.

For those of u...

Table of contents

• Install, manage, and scale a CI/CD build and release system to accelerate your product life cycle
• Foreword
• Contributors
• Preface
• 1 Jenkins Infrastructure with TLS/SSL and Reverse Proxy
• 2 Jenkins with Docker on HTTPS on AWS and inside a Corporate Firewall
• 3 GitOps-Driven CI Pipeline with GitHub
• 4 GitOps-Driven CD Pipeline with Docker Hub and More Jenkinsfile Features
• 5 Headfirst AWS for Jenkins
• 6 Jenkins Configuration as Code (JCasC)
• 7 Backup and Restore and Disaster Recovery
• 8 Upgrading the Jenkins Controller, Agents, and Plugins
• 9 Reducing Bottlenecks
• 10 Shared Libraries
• 11 Script Security
• Index
• Why subscribe?
• Other Books You May Enjoy