DevOps for Serverless Applications

Design, deploy, and monitor your serverless applications using DevOps practices

Shashikant Bangera

• Publisher: Packt Publishing
• Year: 2018
• Print ISBN: 9781788623445
• eBook ISBN: 9781788625661
• Edition: 1
• Topic: Computer Science
• Subtopic: Cloud Computing
• Index: Computer Science

Set up complete CI and CD pipelines for your serverless applications using DevOps principles

Key Features

• Understand various services for designing serverless architecture
• Build CD pipelines using various cloud providers for your serverless applications
• Implement DevOps best practices when building serverless applications

Book Description

Serverless applications are becoming very popular among developers and are generating a buzz in the tech market. Many organizations struggle with the effective implementation of DevOps with serverless applications. DevOps for Serverless Applications takes you through different DevOps-related scenarios to give you a solid foundation in serverless deployment.

You will start by understanding the concepts of serverless architecture and development, and why they are important. Then, you will get to grips with the DevOps ideology and gain an understanding of how it fits into the Serverless Framework. You'll cover deployment framework building and deployment with CI and CD pipelines for serverless applications. You will also explore log management and issue reporting in the serverless environment. In the concluding chapters, you will learn important security tips and best practices for secure pipeline management.

By the end of this book, you will be in a position to effectively build a complete CI and CD delivery pipeline with log management for serverless applications.

What you will learn

• Explore serverless fundamentals and effectively combine them with DevOps
• Set up CI and CD with AWS Lambda and other popular Serverless service providers with the help of the Serverless Framework
• Perform monitoring and logging with serverless applications
• Set up a dynamic dashboard for different service providers
• Discover best practices for applying DevOps to serverless architecture
• Understand use cases for different serverless architectures

Who this book is for

DevOps for Serverless Applications is for DevOps engineers, architects, or anyone interested in understanding the DevOps ideology in the serverless world. You will learn to use DevOps with serverless and apply continuous integration, continuous delivery, testing, logging, and monitoring with serverless.

Applying DevOps to AWS Lambda Applications

Let's briefly look at what we learned about AWS Lambda functions. Amazon Web Services was the first web service platform to launch the serverless computing module Lambda, written in Lambda functions. Lambda functions are stateless and have no affinity with the underlying infrastructure. Lambda functions are executed in response to the events. These events could be an HTTP request, a change of the data in the S3 bucket, a change in a DynamoDB table, or a change in Kinesis or SNS. Lambda functions replicate faster in response to events, and descale as the number of events goes down.

In this chapter, we will be covering various methods of deploying Lambda functions, looking at how we can painlessly deploy to multiple environments, unit test, system test, and integration test the Lambda function. We will also learn various deployment best practices and go through a few example recipes using these best practices. We will see how we can manage the AWS Lambda logs and move them to the ELK stack.

We will explore the following topics in this chapter:

• Manual deployment of a Lambda function
• AWS Lambda with DevOps
• Serverless with CodeStar
• Blue and green deployment with AWS Lambda
• The GitHub and Jenkins pipeline using Serverless Framework
• Setting up Jenkins for a serverless application
• Unit testing a deployed application
• Integrating CloudWatch with ELK

I will be creating a simple application that is used in our day-to-day work. The application is a thumbnail creator, and is a Node.js application that uses two S3 buckets. One bucket is for uploading the actual images and the other is for the thumbnails. The moment an image is uploaded into the images bucket, an event is triggered that calls a function to resize the image and upload it to the thumbnails bucket. We will first look at how we can manually execute this sequence of events, and then we will learn how we can streamline the process by automating the deployment process. In the section dealing with DevOps, we will talk about setting up an assembly line with the development environment, automated testing and deployment, applying a CI/CD pipeline, logging, and monitoring.

Manual deployment of a Lambda function

The Node.js Lambda application that we will be using here is already part of AWS tutorials. We will learn how to create, deploy, and execute a Lambda application via the AWS portal. The prerequisite for this tutorial is for you to have an AWS account; we will be using a free AWS subscription throughout this chapter. The next step is to set up AWS CLI.

You can create an AWS free account and an AWS CLI through the following links:
https://portal.aws.amazon.com/billing/signup#/start
https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html

Go through the following steps:

1. Once the AWS account and CLIs are in place, sign in to AWS Console (https://aws.amazon.com/console/), and then we will create an IAM user with the name adminuser by logging into your AWS account then either clicking on the IAM link or searching for the link through the services:

2. Then we click on the Users link on the left-hand side, where a new page will open. Then we click on Add User. Add a user with the name adminuser and select both the access type Programmatic access and AWS Management Console access. In the console's Password field, add your custom password, uncheck the Require password reset checkbox, and then click Next:Permissions. Let's create a group by clicking on the Create Group button. We will give the group the name of administrators. Next, let's select the AdminstrativeAccess checkbox to provide full access to the group and then click on Create Group. Now that we have a group created, let's click on Next:Review. Then, we will review the user, which the user have created and has been added to the administrator group. Now click on the Create User button. Once the user is created, we should be able to see the user in the list, as shown in the following screenshot:

We have created a user with administrative rights just for our tutorials, but in the real world the role and policies will be more restricted for the sake of security.

3. We need to create two buckets in AWS S3. The buckets need to be created through the adminuser login, so let's log in to AWS Console using the new user that we have created. Click on the adminuser and select the Security credentials tab. Next, let's copy the Console URL for logging in, then open it on the new browser tab. Feed in the username and password for the new user that we have created and click on Sign In. Once you are logged in, search for S3 in the AWS Services. Then go to S3 Console Management and click on the Create bucket button, as shown in the following screenshot:

Let's add a unique bucket and region name of US East by default. The two buckets should be named Source and SourceResized. Source is the placeholder name that should be replaced with the actual bucket name—for example, my-image-bucket76 and my-image-bucket76resized. So, my-image-bucket76 will be the source bucket and my-image-bucket76resized will be the target bucket, as shown in the following screenshot:

The bucket name must be unique, as AWS allows only universally unique bucket names.

4. Once both the buckets are successfully created, we can upload an image for the Lambda function to resize and push to the resized bucket. Let's upload a JPG image into the my-source-bucket76 source. Click on the bucket name, then upload an image to this bucket. This will redirect you to the bucket page. Click on the Upload button and a popup will pop up. Then, select Add files to browse for an image file from the local directory and then upload the image to the S3 bucket.
5. The next step is to create a Lambda function and run it manually. Here, we have to first follow three steps to create the deployment package, and then create the execution role (IAM role) and a Lambda function and test them. The deployment package is a ZIP file containing the Lambda function and its dependencies:
   • Let's create a deployment package. I will be using Node.js as the language for this practice application, but we can use Java and Python as well (it depends on the developer's preference).
   • A prerequisite for creating this deployment package is to ...

Table of contents

• Title Page
• Copyright and Credits
• Dedication
• Packt Upsell
• Contributors
• Preface
• Introducing Serverless
• Understanding Serverless Frameworks
• Applying DevOps to AWS Lambda Applications
• DevOps with Azure Functions
• Integrating DevOps with IBM OpenWhisk
• DevOps with Google Functions
• Adding DevOps Flavor to Kubeless
• Best Practices and the Future of DevOps with Serverless
• Use Cases and Add-Ons
• DevOps Trends with Serverless Functions
• Other Books You May Enjoy