Software Architecture Patterns for Serverless Systems

John Gilbert

• Publisher: Packt Publishing
• Year: 2021
• eBook ISBN: 9781800200739
• Edition: 1
• Topic: Computer Science
• Subtopic: Entreprise Applications
• Index: Computer Science

A professional's guide to solving complex problems while designing modern softwareKey Features• Learn best practices for designing enterprise-grade software systems from a seasoned CTO• Deeper your understanding of system reliability, maintainability, and scalability• Elevate your skills to a professional level by learning the most effective software design patterns and architectural conceptsBook DescriptionAs businesses are undergoing a digital transformation to keep up with competition, it is now more important than ever for IT professionals to design systems to keep up with the rate of change while maintaining stability. This book takes you through the architectural patterns that power enterprise-grade software systems and the key architectural elements that enable change (such as events, autonomous services, and micro frontends), along with showing you how to implement and operate anti-fragile systems. First, you'll divide up a system and define boundaries so that your teams can work autonomously and accelerate innovation. You'll cover low-level event and data patterns that support the entire architecture, while getting up and running with the different autonomous service design patterns. Next, the book will focus on best practices for security, reliability, testability, observability, and performance. You'll combine all that you've learned and build upon that foundation, exploring the methodologies of continuous experimentation, deployment, and delivery before delving into some final thoughts on how to start making progress. By the end of this book, you'll be able to architect your own event-driven, serverless systems that are ready to adapt and change so that you can deliver value at the pace needed by your business.What you will learn• Explore architectural patterns to create anti-fragile systems that thrive with change• Focus on DevOps practices that empower self-sufficient, full-stack teams• Build enterprise-scale serverless systems• Apply microservices principles to the frontend• Discover how SOLID principles apply to software and database architecture• Create event stream processors that power the event sourcing and CQRS pattern• Deploy a multi-regional system, including regional health checks, latency-based routing, and replication• Explore the Strangler pattern for migrating legacy systemsWho this book is forThis book is for software architects who want to learn more about different software design patterns and best practices. This isn't a beginner's manual – you'll need an intermediate level of programming proficiency and software design to get started. You'll get the most out of this software design book if you already know the basics of the cloud, but it isn't a prerequisite.

Chapter 5: Turning the Cloud into the Database

In Chapter 4, Trusting Facts and Eventual Consistency, we covered the event hub and event sourcing patterns and learned how they create an outbound bulkhead that protects upstream services from downstream outages. Now we turn our attention to data architecture and how to reshape it to create inbound bulkheads that protect downstream services from upstream outages. Together, these bulkheads fortify the boundaries of autonomous services and give teams the confidence to forge ahead with changes, knowing that the boundaries will help control the blast radius when things go wrong.

In this chapter, we're going to cover the following main topics:

• Escaping data's gravity
• Embracing the data life cycle
• Turning the database inside out
• Dissecting the CQRS pattern
• Keeping data lean
• Implementing idempotence and order tolerance
• Modeling data for operational performance
• Leveraging change data capture
• Replicating across regions
• Observing resource metrics
• Redacting sensitive data

Escaping data's gravity

In Chapter 1, Architecting for Innovation, we saw that the role of architecture is to enable change so that teams can continuously experiment and discover the best solutions for their users. When we think about architecture, we tend to focus on how we organize and arrange the source code. However, a system's data is arguably its most valuable asset and simultaneously its biggest barrier to change. This is because data has gravity. As a system grows and evolves, so too does its data and its data's structure. We must spend as much or more effort on how we organize and arrange our data so that it does not succumb to similar forces as source code and become brittle, inflexible, and impossible to maintain.

When these deficiencies are combined with the sheer volume of data, the weight of our data becomes an intractable force that prevents teams from moving forward. This impact on a system's ability to grow and evolve is a measure of the data's gravity. To fight data gravity and ultimately escape its pull, we must first understand the forces that so negatively impact our ability to change a system's data architecture.

Let's take a look at how competing demands, insufficient capacity, and intractable volumes influence a system's data gravity.

Competing demands

In Chapter 3, Taming the Presentation Tier, we saw how the presentation tier has acted like a pendulum, oscillating back and forth between client-side and server-side rendering. There is a pendulum in motion at the data tier as well, but it is moving much more slowly between granular and monolithic datastores. In Chapter 1, Architecting for Innovation, we saw how the isolation of information silos drove our industry toward monolithic architecture. However, monolithic databases have proven to be equally deficient, because the force of their data gravity impedes innovation.

In Chapter 2, Defining Boundaries and Letting Go, we found that people (that is, actors) are the driving force behind change. The Single Responsibility Principle (SRP) tells us that each module must be responsible to one and only one actor. But we tend to ignore this principle at the data tier in monolithic databases. In fact, we encourage the reuse of database tables.

This is similar to the topic of whether the presentation tier should live on the client side or the server side. It takes a lot of discipline not to couple business logic with presentation logic when the code resides in the same repository. The same holds true for monolithic databases, because all the tables are easily accessible by all modules using the same database.

Furthermore, relation modeling teaches us to normalize database schemas to the third degree (that is, third normal form) so that there is no duplicate data. This has helped foster the belief that all duplication of data is bad. In other words, it incentivizes the sharing of tables between modules.

Each module that accesses a shared table likely represents a different actor. Each places competing demands on the structure of the database schema. Over time, the schema grows and contorts to accommodate the different requirements. This results in hidden dependencies between the modules. A misunderstanding of these hidden dependencies can lead to mistakes that cause system outages.

Ultimately, these competing demands reduce team confidence, and the force of data gravity starts to grow as we resist making changes.

Insufficient capacity

It's a classic story. You put an application into production, and it performs great for months. Then, one day, it just slows down. After wracking your brain for days, you learn that your application is no longer the sole owner of the database infrastructure. There are now a dozen or so applications competing for the same limited database resources.

This scenario is typical, because owning and operating a database, with all the safeguards in place, is not easy. Our data is our most important asset, and we must protect it. But hardening a database for security and redundancy takes skill, and database administrators are overworked. So, there is a disincentive for running more and more databases. Instead, we make a single database support more and more applications and we add more and more vertical and horizontal capacity. It never seems to be enough. There is always insufficient capacity, so we keep adding more and more.

But now we have created a monstrosity. The database infrastructure has become incredibly complex. Only a select few database administrators are entrusted with making significant changes. Redundancy is baked into the system, but making changes is perceived as high risk because the wrong mistake could cause an outage for all the applications. So we resist change, and the force of data gravity grows stronger.

Intractable volumes

Having lots of data is generally a good thing. The problem is when and where we use it. Our monolithic databases tend to collect data and never let it go, just in case we need it. Meanwhile, the applications that create and use the data need to evolve to support new requirements. This inevitably impacts the data schemas, so the old data must be converted. As the volume of data grows, the conversion effort becomes more difficult and time consuming.

But the applications no longer use much of this data, so why bother converting it? More than likely, the competing demands for the data have created so many interdependencies that we cannot archive the old data. So, it has to stay where it is and as the data grows, it becomes more and more difficult to govern and manipulate. It becomes intractable. It's a compounding effect. We can't archive it, so we must add more capacity. It's difficult to convert, so we resist change. And the force of data gravity grows ever stronger.

The bottom line is that monolithic databases have no bulkheads. There is no real data autonomy. Sooner or later, data's gravity will cause a system to implode or, worse yet, cause innovation to grind to a halt. We need to escape data's gravity by defining fortified boundaries and splitting up the data.

This is what cloud-native is really all about: leveraging the power of the cloud and turning the cloud into the database. Data life cycle architecture and turning the database inside out are two complimentary approaches to breaking up monolithic databases. Let's start by looking at how we use data over its life cycle.

Embracing data life cycle

We need to break up monolithic databases to ensure that our data's gravity does not impede innovation. A natural inclination is to divide a database into sets of related tables and wrap each of these new databases with a data access service. However, this approach is an anti-pattern, as it does not address the problem of competing demands. It just moves data gravity to a service layer and breaks one big problem into multiple smaller problems. It will delay the onset of the force of data gravity, but each database will eventually succumb. We need a better approach.

In Chapter 2, Defining Boundaries and Letting Go, we covered multiple approaches for discovering the boundaries between autonomous subsystems and we introduced a set of autonomous service patterns for decomposing a subsystem into services. For breaking up monolithic databases, the most applicable of these approaches and patterns are the data life cycle architecture and the Backend for Frontend (BFF) pattern.

Data Life Cycle Archit...

Table of contents

• B15957_FM_Final_ASB_ePub
• B15957_TOC_Final_ASB_ePub
• B15957_Preface_Final_ASB_ePub
• B15957_Section 1_Final_ASB_ePub
• B15957_01_Final_ASB_ePub
• B15957_02_Final_ASB_ePub
• B15957_Section 2_Final_ASB_ePub
• B15957_03_Final_ASB_ePub
• B15957_04_Final_ASB_ePub
• B15957_05_Final_ASB_ePub
• B15957_06_Final_ASB_ePub
• B15957_07_Final_ASB_ePub
• B15957_08_Final_ASB_ePub
• B15957_Section 3_Final_ASB_ePub
• B15957_09_Final_ASB_ePub
• B15957_10_Final_ASB_ePub
• B15957_BackMatter_Final_ASB_ePub