Lean Architecture

for Agile Software Development

James O. Coplien, Gertrud Bjørnvig, James O. Coplien

• Publisher: Wiley
• Year: 2011
• Print ISBN: 9780470684207
• eBook ISBN: 9780470970133
• Edition: 1
• Topic: Informatica
• Subtopic: Sviluppo software

More and more Agile projects are seeking architectural roots as they struggle with complexity and scale - and they're seeking lightweight ways to do it

• Still seeking? In this book the authors help you to find your own path
• Taking cues from Lean development, they can help steer your project toward practices with longstanding track records
• Up-front architecture? Sure. You can deliver an architecture as code that compiles and that concretely guides development without bogging it down in a mass of documents and guesses about the implementation
• Documentation? Even a whiteboard diagram, or a CRC card, is documentation: the goal isn't to avoid documentation, but to document just the right things in just the right amount
• Process? This all works within the frameworks of Scrum, XP, and other Agile approaches
  

CHAPTER 1

Introduction

We are changing the Earth more rapidly than we are understanding it.

– Peter Vitousek et al. quoted in The Clock of the Long Now, p. 9.

A proper book isn’t just a collection of facts or even of practices: it reflects a cause and a mission. In the preface we couched this book in a broad context of social responsibility. Just as the motivation section (goal in context, summary, or whatever else you call it) in a use case helps the analyst understand requirements scenarios, this chapter might shed light on the ones that follow. It describes our philosophy behind the book and the way we present the ideas to you. If you’re tempted to jump to a whirlwind tour of the book’s contents, you might proceed to Chapter 2. However, philosophy is as important as the techniques themselves in a Lean and Agile world. We suggest you read through the introduction at least once, and tuck it away in your memory as background material for the other chapters that will support your day-to-day work.

1.1 The Touchstones: Lean and Agile

Lean and Agile are among the most endearing buzzwords in software today, capturing the imagination of management and nerds alike. Popular management books of the 1990s (Womack et al 1991) coined the term Lean for the management culture popularized by the Japanese auto industry, and which can be traced back to Toyota where it is called The Toyota Way. In vernacular English, minimal is an obvious synonym for Lean, but to link lean to minimalism alone is misleading.

Lean’s primary focus is the enterprise value stream. Lean grabs the consumer world and pulls it through the value stream to the beginnings of development, so that every subsequent activity adds value. Waste in production reduces value; constant improvement increases value. In Western cultures managers often interpret Lean in terms of its production practices: just-in-time, end-to-end continuous flow, and reduction of inventory. But its real heart is The Lean Secret: an “all hands on deck” mentality that permeates every employee, every manager, every supplier, and every partner. Whereas the Agile manifesto emphasizes customers, Lean emphasizes stakeholders – with everybody in sight being a stakeholder.

Lean architecture and Agile feature development aren’t about working harder. They’re not about working “smarter” in the academic or traditional computer science senses of the word “smart.” They are much more about focus and discipline, supported by common-sense arguments that require no university degree or formal training. This focus and discipline shines through in the roots of Lean management and in many of the Agile values.

We can bring that management and development style to software development. In this book, we bring it to software architecture in particular. Architecture is the big-picture view of the system, keeping in mind that the best big pictures need not be grainy. We don’t feel a need to nail down a scientific definition of the term; there are too many credible definitions to pick just one. For what it’s worth, the IEEE defines it this way:

… The fundamental organization of a system embodied in its components, their relationships to each other, and to the environment and the principles guiding its design and evolution. (IEEE1471 2007)

Grady Booch gives us this simple definition:

Architecture represents the significant design decisions that shape a system, where significant is measured by cost of change. (Booch 2006)

That isn’t too bad. But more generally, we define architecture as the form of a system, where the word form has a special meaning that we’ll explore a bit later. For now, think of it as relating to the first three components of the IEEE definition. No matter how we care to define it, software architecture should support the enterprise value stream even to the extent that the source code itself should reflect the end user mental model of the world. We will deliver code just in time instead of stockpiling software library warehouses ahead of time. We strive towards the practice of continuous flow.

Each of these practices is a keystone of Lean. But at the heart of Lean architecture is the team: the “all hands on deck” mentality that everyone is in some small part an architect, and that everyone has a crucial role to play in good project beginnings. We want the domain experts (sometimes called the architects) present as the architecture takes shape, of course. However, the customer, the developer, the testers, and the managers should also be fully present at those beginnings.

This may sound wasteful and may create a picture of chaotic beginnings. However, one of the great paradoxes of Lean is that such intensity at the beginning of a project, with heavy iteration and rework in design, actually reduces overall life cycle cost and improves product quality. Apply those principles to software, and you have a lightweight up-front architecture. Lightweight means that we reduce the waste incurred by rework (from inadequate planning), unused artifacts (such as comprehensive documentation and speculative code), and wait states (as can be caused by the review life cycle of architecture and design documents, or by handoffs between functional teams).

Software folks form a tribe of sorts (Nani 2006) that holds many beliefs, among them that architecture is hard. The perception comes in part from architecture’s need for diverse talents working together, compounded by the apparently paradoxical need to find the basic form of something that is essentially complex. Even more important, people confuse “takes a long time” with “hard.” That belief in turn derives from our belief in specialization, which becomes the source of handoffs: the source of the delays that accumulate into long intervals that makes architecture look hard. We tend to gauge our individual uncertainty and limited experience in assessing the difficulty of design, and we come up short, feeling awkward and small rather than collaborative and powerful. Architecture requires a finesse and balance that dodges most silver bullets. Much of that finesse comes with the Lean Secret: the takes-a-long-time part of hard becomes softer when you unite specialists together in one room: everybody, all together, from early on. We choose to view that as hard because, well, that’s how it’s always been, and perhaps because we believe in individuals first and interactions second.

Neither Lean nor Agile alone make architecture look easy. However, architecture needn’t be intrinsically hard. Lean and Agile together illuminate architecture’s value. Lean brings careful up-front planning and “everybody, all together, from early on” to the table, and Agile teaches or reminds us about feedback. Together they illuminate architecture’s value: Lean, for how architecture can reduce waste, inconsistency, and irregular development; and Agile, for how end user engagement and feedback can drive down long-term cost. Putting up a new barn is hard, too. As Grandpa Harry used to say, many hands make light work, and a 19^th-century American farm neighborhood could raise a new barn in a couple of days. So can a cross-functional team greatly compress the time, and therefore the apparent difficulty, of creating a solid software architecture.

Another key Lean principle is to focus on long-term results (Liker 2004, pp. 71–84). Lean architecture is about doing what’s important now that will keep you in the game for the long term. It is nonetheless important to contrast the Lean approach with traditional approaches such as “investing for the future.” Traditional software architecture reflects an investment model. It capitalizes on heavyweight artifacts in software inventory and directs cash flow into activities that are difficult to place in the customer value stream. An industry survey of projects with ostensibly high failure rates (as noted in Glass (2006), which posits that the results of the Standish survey may be rooted in characteristically dysfunctional projects) found that 70% of the software they build is never used (Standish Group 1995).

Lean architecture carefully slices the design space to deliver exactly the artifacts that can support downstream development in the long term. It avoids wasteful coding that can better be written just after demand for it appears and just before it generates revenues in the market. From the programmer’s perspective, it provides a way to capture crucial design concepts and decisions that must be remembered throughout feature production. These decisions are captured in code that is delivered as part of the product, not as extraneous baggage that becomes irrelevant over time.

With such Lean foundations in place, a project can better support Agile principles and aspire to Agile ideals. If you have all hands on deck, you depend more on people and interactions than on processes and tools. If you have a value stream that drives you without too many intervening tools and processes, you have customer engagement. If we reflect the end user mental model in the code, we are more likely to have working software. And if the code captures the form of the domain in an uncluttered way, we can confidently make the changes that make the code serve end user wants and needs.

This book is about a Lean approach to domain architecture that lays a foundation for Agile software change. The planning values of Lean do not conflict with the inspect-and-adapt principles of Agile: allocated to the proper development activities, each supports the other in the broader framework of development. We’ll revisit that contrast in a little while (Section 1.4), but first, let’s investigate each of Lean Architecture and Agile Production in more detail.

1.2 Lean Architecture and Agile Feature Development

The Agile Manifesto (Beck et al 2001) defines the principles that underlie the Agile vision, and the Toyota Way (Liker 2004) defines the Lean vision. This book offers a vision of architecture in an organization that embraces these two sets of ideals. The Lean perspective focuses on how we develop the overall system form by drawing on experience and domain knowledge. The Agile perspective focuses on how that informed form helps us respond to change, and sometimes even to plan for it. How does that vision differ from the classic, heavyweight architectural practices that dominated object-oriented development in the 1980s? We summarize the differences in Table 1-1.

Table 1-1 What is Lean Architecture?

Lean Architecture	Classic Software Architecture
Defers engineering	Includes engineering
Gives the craftsman “wiggle room” for change	Tries to limit large changes as “dangerous” (fear change?)
Defers implementation (delivers lightweight APIs and descriptions of relationships)	Includes much implementation (platforms, libraries) or none at all (documentation only)
Lightweight documentation	Documentation-focused, to describe the implementation or compensate for its absence
People	Tools and notations
Collective planning and cooperation	Specialized planning and control
End user mental model	Technical coupling and cohesion

Classic software architecture tends to embrace engineering concerns too strongly and too early. Agile architecture is about form, and while a system must obey the same laws that apply to engineering when dealing with form, we let form follow proven experience instead of being driven by supposedly scientific engineering rationales. Those will come soon enough.

This in turn implies that the everyday developers should use their experience to tailor the system form as new requirements emerge and as they grow in understanding. Neither Agile nor Lean gives coders wholesale license to ravage the system form, but both honor the value of adaptation. Classic architecture tends to be fearful of large changes, so it focuses on incremental changes only to existing artifacts: adding a new derived class is not a transformation of form (architecture), but of structure (implementation). In our combined Lean/Agile approach, we reduce risk by capturing domain architecture, or basic system form, in a low-overhead way. Furthermore, the architecture encourages new forms in those parts of the system that are likely to ch...

Table of contents

• Cover
• Title page
• Copyright page
• Dedication
• Publisher’s Acknowledgments
• About the Authors
• Preface
• 1 Introduction
• 2 Agile Production in a Nutshelln
• 3 Stakeholder Engagement
• 4 Problem Definition
• 5 What the System Is, Part 1: Lean Architecture
• 6 What the System Is, Part 2: Coding It Up
• 7 What the System Does: System Functionality
• 8 Coding It Up: Basic Assembly
• 9 Coding it Up: The DCI Architecture
• 10 Epilog
• Appendix A Scala Implementation of the DCI Account Example
• Appendix B Account Example in Python
• Appendix C Account Example in C#
• Appendix D Account Example in Ruby
• Appendix E Qi4j
• Appendix F Account Example in Squeak
• Bibliography
• Index
• End User License Agreement