Hands-On Software Engineering with Golang

Move beyond basic programming to design and build reliable software with clean code

Achilleas Anagnostopoulos

• Publisher: Packt Publishing
• Year: 2020
• Print ISBN: 9781838554491
• eBook ISBN: 9781838550240
• Edition: 1
• Topic: Computer Science
• Subtopic: Programming Languages
• Index: Computer Science

Explore software engineering methodologies, techniques, and best practices in Go programming to build easy-to-maintain software that can effortlessly scale on demand

Key Features

• Apply best practices to produce lean, testable, and maintainable Go code to avoid accumulating technical debt
• Explore Go's built-in support for concurrency and message passing to build high-performance applications
• Scale your Go programs across machines and manage their life cycle using Kubernetes

Book Description

Over the last few years, Go has become one of the favorite languages for building scalable and distributed systems. Its opinionated design and built-in concurrency features make it easy for engineers to author code that efficiently utilizes all available CPU cores.

This Golang book distills industry best practices for writing lean Go code that is easy to test and maintain, and helps you to explore its practical implementation by creating a multi-tier application called Links 'R' Us from scratch. You'll be guided through all the steps involved in designing, implementing, testing, deploying, and scaling an application. Starting with a monolithic architecture, you'll iteratively transform the project into a service-oriented architecture (SOA) that supports the efficient out-of-core processing of large link graphs. You'll learn about various cutting-edge and advanced software engineering techniques such as building extensible data processing pipelines, designing APIs using gRPC, and running distributed graph processing algorithms at scale. Finally, you'll learn how to compile and package your Go services using Docker and automate their deployment to a Kubernetes cluster.

By the end of this book, you'll know how to think like a professional software developer or engineer and write lean and efficient Go code.

What you will learn

• Understand different stages of the software development life cycle and the role of a software engineer
• Create APIs using gRPC and leverage the middleware offered by the gRPC ecosystem
• Discover various approaches to managing package dependencies for your projects
• Build an end-to-end project from scratch and explore different strategies for scaling it
• Develop a graph processing system and extend it to run in a distributed manner
• Deploy Go services on Kubernetes and monitor their health using Prometheus

Who this book is for

This Golang programming book is for developers and software engineers looking to use Go to design and build scalable distributed systems effectively. Knowledge of Go programming and basic networking principles is required.

Section 1: Software Engineering and the Software Development Life Cycle

The objective of part one is to familiarize you with the concept of software engineering, the stages of the software development life cycle, and the various roles of software engineers.

This section comprises the following chapter:

• Chapter 1, A Bird's-Eye View of Software Engineering

A Bird's-Eye View of Software Engineering

"Hiring people to write code to sell is not the same as hiring people to design and build durable, usable, dependable software."

- Larry Constantine ^[6]

Through the various stages of my career, I have met several people that knew how to code; people whose skill level ranged from beginner to, what some would refer to as, guru. All those people had different backgrounds and worked for both start-ups and large organizations. For some, coding was seen as a natural progression from their CS studies, while others turned to coding as part of a career change decision.

Regardless of all these differences, all of them had one thing in common: when asked to describe their current role, all of them used the term software engineer. It is quite a common practice for job candidates to use this term in their CVs as the means to set themselves apart from a globally distributed pool of software developers. A quick random sampling of job specs published online reveals that a lot of companies – and especially high-profile start-ups – also seem to subscribe to this way of thinking, as evidenced by their search for professionals to fill software engineering roles. In reality, as we will see in this chapter, the term software engineer is more of an umbrella term that covers a wide gamut of bespoke roles, each one combining different levels of software development expertise with specialized skills pertaining to topics such as system design, testing, build tools, and operations management.

So, what is software engineering and how does it differ from programming? What set of skills should a software engineer possess and which models, methodologies, and frameworks are at their disposal for facilitating the delivery of complex pieces of software? These are some of the questions that will be answered in this chapter.

This chapter covers the following topics:

• A definition of software engineering
• The types of software engineering roles that you may encounter in contemporary organizations
• An overview of popular software development models and which one to select based on the project type and requirements

What is software engineering?

Before we dive deeper into this chapter, we need to establish an understanding of some of the basic terms and concepts around software engineering. For starters, how do we define software engineering and in what ways does it differ from software development and programming in general? To begin answering this question, we will start by examining the formal definition of software engineering, as published in IEEE's Standard Glossary of Software Engineering Terminology ^[7]:

"Software engineering is defined as the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software."

The main takeaway from this definition is that authoring code is just one of the many facets of software engineering. At the end of the day, any capable programmer can take a well-defined specification and convert it into a fully functioning program without thinking twice about the need to produce clean and maintainable code. A disciplined software engineer, on the other hand, would follow a more systematic approach by applying common design patterns to ensure that the produced piece software is extensible, easier to test, and well documented in case another engineer or engineering team assumes ownership of it in the future.

Besides the obvious requirement for authoring high-quality code, the software engineer is also responsible for thinking about other aspects of the systems that will be built. Some questions that the software engineer must be able to answer include the following:

• What are the business use cases that the software needs to support?
• What components comprise the system and how do they interact with each other?
• Which technologies will be used to implement the various system components?
• How will the software be tested to ensure that its behavior matches the customer's expectations?
• How does load affect the system's performance and what is the plan for scaling the system?

To be able to answer these questions, the software engineer needs a special set of skills that, as you are probably aware, go beyond programming. These extra responsibilities and required skills are the main factors that differentiate a software engineer from a software developer.

Types of software engineering roles

As we discussed in the previous section, software engineering is an inherently complex, multi-stage process. In an attempt to manage this complexity, organizations around the world have invested a lot of time and effort over the years to break the process down into a set of well-defined stages and train their engineering staff to efficiently deal with each stage.

Some software engineers strive to work across all the stages of the Software Development Life Cycle (SDLC), while others have opted to specialize in and master a particular stage of the SDLC. This gave rise to a variety of software engineering roles, each one with a different set of responsibilities and a required set of skills. Let's take a brief look at the most common software engineering roles that you may encounter when working with both small- and large-sized organizations.

The role of the software engineer (SWE)

The software engineer (SWE) is the most common role that you are bound to interact with in any organization, regardless of its size. Software engineers play a pivotal role not only in designing and building new pieces of software, but also in operating and maintaining existing and legacy systems.

Depending on their experience level and technical expertise, SWEs are classified into three categories:

• Junior engineer: A junior engineer is someone who has recently started their software development career and lacks the necessary experience to build and deploy production-grade software. Companies are usually keen on hiring junior engineers as it allows them to keep their hiring costs low. Furthermore, companies often pair promising junior engineers with senior engineers in an attempt to grow them into mid-level engineers and retain them for longer.
• Mid-level engineer: A typical mid-level engineer is someone who has at least three years of software development experience. Mid-level engineers are expected to have a solid grasp of the various aspects of the software development life cycle and are the ones who can exert a significant impact on the amount of code that's produced for a particular project. To this end, they not only contribute code, but also review and offer feedback to the code that's contributed by other team members.
• Senior engineer: This class of engineer is well-versed in a wide array of disparate technologies; their breadth of knowledge makes them ideal for assembling and managing software engineering teams, as well as serving as mentors and coaches for less senior engineers. From their years of experience, senior engineers acquire a deep understanding of a particular business domain. This trait allows them to serve as a liaison between their teams and the other, technical or non-technical, business stakeholders.

Another way to classify software engineers is by examining the main focus of their work:

• Frontend engineers work exclusively on software that customers interact with. Examples of frontend work include the UI for a desktop application, a single-page web application for a software as a service (SaaS) offering, and a mobile application running on a phone or other smart device.
• Backend engineers specialize in building the parts of a system that implement the actual business logic and deal with data modeling, validation, storage, and retrieval.
• Full stack engineers are developers who have a good understanding of both frontend and backend technologies and no particular preference of doing frontend or backend work. This class of developers is more versatile as they can easily move between teams, depending on the project requirements.

The role of the software development engineer in test (SDET)

The software development engineer in test (SDET) is a role whose origins can be traced back to Microsoft's engineering teams. In a nutshell, SDETs are individuals who, just like their SWE counterparts, take part in software development, but their primary focus lies in software testing and performance.

An SDET's primary responsibility is to ensure that the development team produces high-quality software that is free from defects. A prerequisite for achieving this goal is to be cognizant of the different types of approaches to testing software, including, but not limited to, unit testing, integration testing, white/black-box testing, end-to-end/acceptance testing, and chaos testing. We will be discussing all of these testing approaches in more detail in the following chapters.

The main tool that SDETs use to meet their goals is testing automation. Development teams can iterate much faster when a Continuous Integration (CI) pipeline is in place t...

Table of contents

• Title Page
• Copyright and Credits
• Dedication
• About Packt
• Contributors
• Preface
• Section 1: Software Engineering and the Software Development Life Cycle
• A Bird's-Eye View of Software Engineering
• Section 2: Best Practices for Maintainable and Testable Go Code
• Best Practices for Writing Clean and Maintainable Go Code
• Dependency Management
• The Art of Testing
• Section 3: Designing and Building a Multi-Tier System from Scratch
• The Links 'R'; Us Project
• Building a Persistence Layer
• Data-Processing Pipelines
• Graph-Based Data Processing
• Communicating with the Outside World
• Building, Packaging, and Deploying Software
• Section 4: Scaling Out to Handle a Growing Number of Users
• Splitting Monoliths into Microservices
• Building Distributed Graph-Processing Systems
• Metrics Collection and Visualization
• Epilogue
• Assessments
• Other Books You May Enjoy