Hands-On RESTful Web Services with Go

Develop elegant RESTful APIs with Golang for microservices and the cloud, 2nd Edition

Naren Yellavula

• Publisher: Packt Publishing
• Year: 2020
• eBook ISBN: 9781838647544
• Edition: 2
• Topic: Computer Science
• Subtopic: Programming Languages
• Index: Computer Science

Design production-ready, testable, and maintainable RESTful web services for the modern web that scale easily

Key Features

• Employ a combination of custom and open source solutions for application program interface (API) development
• Discover asynchronous API and API security patterns and learn how to deploy your web services to the cloud
• Apply design patterns and techniques to build reactive and scalable web services

Book Description

Building RESTful web services can be tough as there are countless standards and ways to develop API. In modern architectures such as microservices, RESTful APIs are common in communication, making idiomatic and scalable API development crucial. This book covers basic through to advanced API development concepts and supporting tools.

You'll start with an introduction to REST API development before moving on to building the essential blocks for working with Go. You'll explore routers, middleware, and available open source web development solutions in Go to create robust APIs, and understand the application and database layers to build RESTful web services. You'll learn various data formats like protocol buffers and JSON, and understand how to serve them over HTTP and gRPC. After covering advanced topics such as asynchronous API design and GraphQL for building scalable web services, you'll discover how microservices can benefit from REST. You'll also explore packaging artifacts in the form of containers and understand how to set up an ideal deployment ecosystem for web services. Finally, you'll cover the provisioning of infrastructure using infrastructure as code (IaC) and secure your REST API.

By the end of the book, you'll have intermediate knowledge of web service development and be able to apply the skills you've learned in a practical way.

What you will learn

• Explore the fundamentals of API development and web services
• Understand the various building blocks of API development in Go
• Use superior open source solutions for representational state transfer (REST) API development
• Scale a service using microservices and asynchronous design patterns
• Deliver containerized artifacts to the Amazon Web Services (AWS) Cloud
• Get to grips with API security and its implementation

Who this book is for

This book is for all the Go developers who are comfortable with the language and seeking to learn REST API development. Even senior engineers can enjoy this book, as it discusses many cutting-edge concepts, such as building microservices, developing API with GraphQL, using protocol buffers, asynchronous API design, and Infrastructure as a Code. Developers who are already familiar with REST concepts and stepping into the Go world from other platforms, such as Python and Ruby, can also benefit a lot.

Asynchronous API Design

In this chapter, we are going to discuss how to design an asynchronous API for clients. We will look into strategies such as queuing tasks and publish/subscribe paradigms. A synchronous request waits on the server to compute the result. On the other hand, an asynchronous (async) request receives a response immediately with the information about the eventual result. The real world is composed of many synchronous and asynchronous events.

Asynchronous events are very popular in browsers. An async API mimics the same behavior as an event loop in modern browsers. In this chapter, we'll look at the difference between the request types. We'll also write a few clients in Go that can consume an asynchronous API.

In this chapter, we will cover the following topics:

• Understanding sync/async API requests
• Fan-in/fan-out of services
• Delaying API jobs with queuing
• Long-running task design
• Caching strategies for APIs
• Event-driven API

Technical requirements

You will need to install the following software to run the code samples in this chapter:

• OS: Linux (Ubuntu 18.04)/Windows 10/Mac OS X >=10.13
• Go stable version compiler >= 1.13.5
• Dep: A dependency management tool for Go >= 0.5.3
• Docker version >= 18.09.2

You can download the code for this chapter from https://github.com/PacktPublishing/Hands-On-Restful-Web-services-with-Go/tree/master/chapter9. Clone the code and use the code samples in the chapter9 directory.

Understanding sync/async API requests

A synchronous request is an HTTP request that blocks the server until the response is returned. The majority of the services on the web run in this fashion. Nowadays, with the advent of distributed systems and loose coupling, API requests can also be asynchronous. In other words, an asynchronous request returns with information that can be used to fetch the information of a process. These asynchronous requests on a server are closely related to how concurrently the server can execute a job for multiple clients. Let's look at what a synchronous request looks like:

In this type of request, the web server performs all the actions and returns an Immediate Response to the Web client/Mobile client. The drawback of this approach is that if the server takes too much time to render the result, the client is blocked on the server's action.

An asynchronous request instantly returns a response but not with the result. It issues a ticket for finding the status of the requested operation. A client can use that ticket (a different response) to check the status and the final result of the operation:

As shown in the preceding diagram, the client is sending a request to the server and the server returns a response to the client. This response is not something the client can consume instantly. Long-running tasks/jobs can be made asynchronous by the server. The client can then use the received response to find out the status of the job. Once the job is done, either the server can notify the client or the client can poll the result by looking at the status. So far, we have only built a synchronous API. This chapter will discuss its implementation in detail.

In the next section, we'll discuss how APIs can diverge into multiple or submerge into a single call. These techniques are called fan-out and fan-in, respectively.

Fan-in/fan-out of services

Let's take a real-world example of an e-commerce website integrating itself with a third-party payment gateway. Here, the website uses an API from the payment gateway to pop up the payment screen and enters security credentials. At the same time, the website may call another API called analytics to record the attempt of payment. This process of forking a single request into multiple is called fan-out. In the real world, there can be many fan-out services involved in a single client request.

Another example is MapReduce. Map is a fan-in operation, while Reduce is a fan-out operation. A server can fan out a piece of information to the next set of services (API) and ignore the result or can wait until all the responses from those servers are returned. As shown in the following diagram, an incoming request is being multiplexed by the server into two outgoing requests:

This process is a simple fan-out.

Fan-in is an operation where two or more incoming requests converge into a single request. This scenario is how an API aggregates results from multiple backend services and returns the result on the fly to a client. For example, think about a hotel price aggregator or flight ticket aggregator that fetches requested information about multiple hotels or flights from various data providers and displays them. The following diagram shows how a fan-in operation combines multiple requests and prepares a final response that's consumed by a client:

The client can also be a server that serves further clients. As shown in the preceding diagram, the left-side hand server is collecting the responses from Hotel A, Hotel B, and Airline Provider A and preparing another response for a different client. Therefore, fan-in and fan-out operations are not always completely independent of each other. Mostly, it will be a hybrid scenario where both fan-in and fan-out operations fit with each other.

Please remember that the fan-out operation to the next set of servers can be asynchronous too. This may not be true with fan-in requests. A fan-in operation is sometimes called an API call.

In this section, we've seen how fan-in and fan-out work. To use these techniques, we need to know how to implement an asynchronous service (API). In the next section, we'll try to implement such a service using a mechanism called job queuing.

Delaying API jobs with queuing

In synchronous APIs, the blocking code plays a crucial role in preparing the response that is sent to the client. However, in the asynchronous design, non-blocking is key. A queue and workers can be helpful in achieving non-blocking code. A server can have multiple workers running in parallel who can exhaust the contents of a queue and work on them. Whenever a client requests an operation through an asynchronous API, the server can put that request in a job queue, and all the workers can pick up a task whenever their turn comes.

This approach can offload an API server and focus on its business logic instead of getting blocked on parallel/independent tasks such as sending emails, contacting third-party services, and so on.

A few use cases of queuing are as follows:

• Compress images and email the final result
• Automatic back pressuring (limiting the load on the server to predictable amounts)

To explain this concept in detail, let's formulate an example and try to implement it.

Let's develop an ...

Table of contents

• Title Page
• Copyright and Credits
• Dedication
• About Packt
• Contributors
• Preface
• Getting Started with REST API Development
• Handling Routing for our REST Services
• Working with Middleware and RPC
• Simplifying RESTful Services with Popular Go Frameworks
• Working with MongoDB and Go to Create a REST API
• Working with Protocol Buffers and gRPC
• Working with PostgreSQL, JSON, and Go
• Building a REST API Client in Go
• Asynchronous API Design
• GraphQL and Go
• Scaling our REST API Using Microservices
• Containerizing REST Services for Deployment
• Deploying REST Services on Amazon Web Services
• Handling Authentication for our REST Services
• Other Books You May Enjoy