Mastering Puppet 5

Optimize enterprise-grade environment performance with Puppet

Ryan Russell-Yates, Jason Southgate

• Publisher: Packt Publishing
• Year: 2018
• ISBN:
  9781788833325
• Edition: 1
• Topic: Computer Science
• Subtopic: Computer Engineering
• Index: Computer Science

Leverage Puppet 5 for medium to large scale enterprise deployment.

Key Features

• Use and deploy Puppet 5, irrespective the size of your organization
• Scaling, performance improvements, and managing multiple developer requests
• Troubleshooting techniques, tips and tricks to make the most of Puppet 5

Book Description

Puppet is a configuration management system and a language written for and by system administrators to manage a large number of systems efficiently and prevent configuration drift.

The core topics this book addresses are Puppet's latest features and mastering Puppet Enterprise. You will begin by writing a new Puppet module, gaining an understanding of the guidelines and style of the Puppet community. Following on from this, you will take advantage of the roles and profiles pattern, and you will learn how to structure your code. Next, you will learn how to extend Puppet and write custom facts, functions, types, and providers in Ruby, and also use the new features of Hiera 5. You will also learn how to configure the new Code Manager component, and how to ensure code is automatically deployed to (multiple) Puppet servers. Next, you will learn how to integrate Puppet with Jenkins and Git to build an effective workflow for multiple teams, and use the new Puppet Tasks feature and the latest Puppet Orchestrator language extensions. Finally, you will learn how to scale and troubleshoot Puppet.

By the end of the book, you will be able to deal with problems of scale and exceptions in your code, automate workflows, and support multiple developers working simultaneously.

What you will learn

• Solve problems using modules and the roles and profiles pattern
• Extend Puppet with custom facts, functions, types, and providers
• Use Hiera 5 and Code Manager/r10k to separate code from data
• Continuously integrate your code using Jenkins, Git and automated testing
• Use exported resources and the new Puppet Orchestration features
• Explore Puppet Discovery features and their use
• Troubleshoot various parts of the Puppet Enterprise infrastructure
• Scale up and scale out Puppet infrastructure using various techniques

Who this book is for

If you are a system administrator or developer who has used Puppet in production and are looking for ways to easily use Puppet in an enterprise environment, this book is for you. Some knowledge of writing simple configuration management modules would be necessary.

Workflow

In this chapter, we'll discuss the workflow in Puppet. We'll cover what makes a good technical workflow, how to apply that to Puppet, and how to use the Puppet Development Kit (PDK) to improve our workflow. We'll investigate the following qualities of a good workflow: ease of use, rapid feedback, ease of onboarding, and quality control. We'll use Puppet Git repositories to provide a basic Puppet workflow that can be tuned to any system of management. We'll also explore the new PDK released by Puppet, which can improve our workflow.

The following topics will be covered in this chapter:

• Puppet workflow
• Designing a Puppet workflow
• Using the PDK

Puppet workflow

A workflow is a series of processes that work flows through, from initiation to completion. As the Puppet environments become more complex in an organization, a trusted and shared workflow will make sharing work easier. A Puppet workflow should allow us to access code, edit code, test our code, and, eventually, deploy our code back to the Puppet Master. Although it is not required, it is highly recommended that an organization or group of workers adopt a shared workflow. A shared workflow possesses a few main benefits, as follows:

• A measurable ease of use
• Rapid feedback
• Ease of onboarding
• Quality control

Ease of use

The primary reason to design and begin a workflow is to provide for ease of use. A team should design a workflow around their code base, allowing them to understand how to retrieve specific code, how to edit that code, and the impacts of the new edits. A workflow also provides a standardized way of packaging the code, to be delivered and used by the existing code base. Each step in the workflow should be clear, concise, communicated, and repeatable. It is important that everyone on the team understands not only how the workflow works, but why each step of the workflow exists, so that they can troubleshoot and contribute to the workflow, should something change in the organization.

One of the primary benefits of a shared workflow, as opposed to individualized workflows, is the ability to measure the impact of the workflow on the organization. To measure our workflow, we first separate standard and nonstandard units of work. The edits that we make to our code often vary in size and complexity, and are not easy to measure in standard units. On the other hand, code is generally checked out, tested, and deployed in the same way every time, leaving us with a good estimate of how long it will take to go through our workflow, minus the code edits.

If our workflow takes about 30 seconds to clone the code repository, an unknown amount of time to edit code, 5 minutes to run a test, and another 30 seconds to deploy the code in our environment, our workflow, with a single test, will take about 6 minutes. If we have eight members of our team, who each run through this workflow 10 times a day on average, our workflow actually constitutes about 8 hours a week of our combined work (8 x 10 x 6 = 480 minutes, or 8 hours). Cutting this testing time in half reduces our total time as a team spent on the workflow by about 3 1/3 hours per week. Because of this measurable amount of time that can be saved in a workflow, a team should consider optimizing their workflow whenever possible.

Generally, you won't need more than a rough estimate of the time it takes to perform the standard functions of the workflow, but you will need to know which pieces might be performed more than once. With Puppet, a user will likely write, push, and test code more than they will pull it down. You can inspect each piece of the workflow separately and seek to improve a part of the process, but you should consider the ramifications of a change to the rest of the workflow.

Rapid feedback

A good workflow should provide constant feedback to its users. Each step should be clearly defined, with strict pass or fail criteria. For example, Git will warn a user whenever it detects a problem, such as being unable to pull code or push code back to the origin repository. We can extend this with Git commit hooks, both server-side and client-side, which perform checks to ensure that the code is in a proper state before being accepted into an organizational Git server from the local repository. Running Puppet itself within our test criteria, we expect clean and idempotent runs. The Puppet catalog should not produce failing resources, nor should it manage the same resource with every Puppet run.

The time it takes to solve problems with Puppet shrinks as more feedback is provided by a workflow to the engineer. If you work in a workflow that requires pushing code to an environment on the Puppet Master, and you are testing on a true agent, a simple run of puppet parser validate can save a lot of time. The parser validation will quickly tell you if Puppet code can be compiled, rather than what it will do. This simple command can reduce the number of times that we git commit on the code, push it to the Git repository, deploy it to an environment, log in to the test machine, and wait for the Puppet agent to trigger a catalog error. We can even ensure that this command is run before every commit with a precommit Git hook. Automated testing tools, such as RSpec and Beaker, can extend this methodology, and, combined with a CI/CD pipeline (discussed in the next chapter), can provide even more rapid feedback to code developers.

Ease of onboarding

A well-built workflow naturally facilitates the ease of adding new members to a project, whether open source or a part of an organization. A simple tool suite and guide can be invaluable to those new members, and can help them to get over the hurdle of the first commit. Even a simple getting started README can go a long way, if properly maintained. Onboarding new members to a project is costly, and quality workflow can minimize the time spent by the new member. Bringing on new project members also requires some information and time from existing project members. If your project is an ongoing development effort, it's highly likely that you'll have some turnover, and saving time for existing members while shortening the time for new members to reach effectiveness should be a priority in your workflow.

Quality control

A good workflow should always seek to reduce mistakes and increase code quality. Every built-in safety mechanism in a workflow allows a team to iterate over more complex features more quickly. Simple things, such as preventing pushes directly to production branches and basing production environments on semantically versioned code, allow for rapid development, without any worries about toppling critical infrastructure.

The following lists a few examples of workflow improvements designed around security and stability:

• Preventing direct code pushes to production on the control repository
• Preventing direct code pushes to masters on individual modules
• Running Puppet parser validation on all manifests prior to a push back to th...