Chapter 1
Introduction
‘ Software entities are more complex for their size than perhaps any other human construct because no two parts are alike ’
No Silver Bullet, Brooks, 1986
Building software is hard: successfully bringing a new software product from conception to market is harder. Building a successful software company that develops and markets multiple software products is harder still.
The days of ‘build it and they will come’ are over. Simply creating a great piece of software and waiting for the customers to knock on your door no longer works – if it ever did.
Nobody should ever think that building software is easy: some small application may indeed be easy to build, but meaningful software applications and platforms are extremely complicated things to create. However, building software is at least a reasonably well-defined task which isn't true of many of the other things that need to be done. Indeed it isn't even clear what needs doing.
Creating a successful software product, and building a successful software company, involve a myriad of other activities that are far less easy to define and put boundaries around. The first of these is just deciding what it is that should be built. But perhaps the most difficult task of all is bringing all these activities – well-defined, poorly defined and never defined alike – into alignment so that they move towards a common goal. This is akin to an exercise in formation flying in which the individual pilots have their own opinion on where they should be going. The bigger the company, the more complex the environment, the more difficult the task.
Focusing on the customer helps: after all, we all agree the customer comes first (don't we?) But while it might be clear what an individual customer says they want, what they are willing and able to pay for might be something different. When individual customers are aggregated into a market – be it millions, thousands or just dozens – it can be extremely difficult truly to know what the customer wants.
Customers are not homogenous: they want different things, but if you intend to build a product to sell to many of them, you need to determine common needs. Great leaders help, but great leaders are not a panacea. Leadership can give direction and common goals, but if a company can only survive with a truly remarkable leader, then the leader has failed. Leaders lead, but they also need to put in place mechanisms to prevent single points of failure.
Given all these difficulties, and more, is it a any surprise that many choose to focus on something right in front of them? Something here and now? We might call it ‘goal displacement’. Building a company is an engineering task to parallel the construction of the software itself.
The patterns in this book describe many of the recurring problems software companies face and details the solutions found by existing companies. The patterns follow the well-known pattern format: a problem statement, a set of forces that make the problem difficult, a solution, with details of how to build it, the consequences of the solution, and examples and related work.
The Whole
‘No plan survives contact with the enemy.’
General Helmuth von Moltke
Each pattern in this book contains one part of a solution. It may be used in isolation – a component if you like, dropped in to fix an issue. But each pattern also links to other patterns, and together they build towards a complete thing, a whole.
The whole is more than the sum of its parts. The whole is the thing you see when everyone is flying in formation. The whole happens when all the different parts of the organisation – engineering, sales, marketing – are coordinated, aligned, heading in the same direction. In business, making this happen is strategy.
Strategy sets organisational goals, objectives and direction. It is embodied in structures and organisation. It is planned, but it is also dynamic. It is changing in response to events and learning. Strategy looks forward and looks backwards to make sense of events. Strategy may be equated with a plan, something that is determined by a few – top management, consultants, planners – and executed by the many. But the devil is in the detail: things are just too complicated for a comprehensive plan.
While there are elements of forward-looking planning, strategy is equally the result of what happens; it emerges from events and decisions: what happens in the market, what engineers learn as they build the product, what competitors do. Strategy is backwards-looking; it makes sense of what has happened in order to direct the future.
Patterns are ideal for describing strategy, because they share these characteristics. Patterns are descriptions of what happens – like strategy, they emerge from looking backwards, adding to understanding and making sense. They are also forward-looking: they describe what to do.
Used alone, an individual pattern might be strategic, or it might just be something done to fix a problem – that is, tactical. When brought together, when used in a sequence to form a whole, they work strategically. Patterns direct and inform, but they are not a straight-jacket; you can, and should, vary a pattern as you see fit. This is how new patterns come into being.
Patterns describe the problems and the solution, and importantly, they name these things. Perhaps the most important thing a pattern adds to a discussion is a name. Patterns bring a shared vocabulary, which raises the level of the discussion.
Rather than talking about ‘that thing you do when you involve the customer in creating the product but don't create the product only for that customer’ you can say Customer Co-Created Product. This raises the level of the discussion and the abstraction level, it moves the conversation along more quickly and gives everyone, from CEO to lowly software engineer, a common language.
Patterns
‘According to leading management thinkers, the manufacturing, service, and information sectors will be based on knowledge in the coming age, and business organisations will evolve into knowledge creators in many ways.
According to [Peter Drucker] we are entering ‘the knowledge society,’ in which ‘the basic resource’ is no longer capital, or natural resources, or labour, but ‘is and will be knowledge…’
(Nonaka & Takeuchi, 1995)
Patterns are all around us; it is simply a question of whether we see them and whether anyone has documented them. Patterns are recurring events, situations, constructions and ideas. They occur over and over again. Things that happen once are not patterns, but when they recur several times we label them patterns.
A pattern exists for a reason. A pretty design on a tie, on a sheet of wallpaper or on a dress is just that, a pretty design. These patterns exist to decorate people and places; attractiveness is no small matter. When these designs are used again and again, they take on meaning: they are used again and again for a reason.
The pattern on a tie may simply be there to make a businessman look good, but it may also serve to identify him as a member of an association, a clan or an alumni of a college. Similarly, a pattern on a dress may make the woman wearing it look good, but it may also help to identify her or convey authority and legitimacy (think of a nurse). Wallpaper may look nice, but we would choose different wallpaper for a gentleman's club to that for decorating a baby's nursery.
Things recur for reasons; things repeat for a reason.
In the 1970s the architect Christopher Alexander devised the ‘pattern form’ to codify his thinking about architecture. Using patterns, he documented the way in which buildings are located, laid out and constructed in recurring ways for reasons – reasons he came to call forces. Traditionally the logic behind these designs was handed down by word-of-mouth and through participation. Today those who study these things would call this ‘tacit’ knowledge and would say it was communicated through stories and legitimate peripheral participation.
Alexander also believed that this knowledge was being lost. Modern architecture and modern society separates architects and builders from those who use the buildings. So he set about codifying the patterns that made buildings the way they are.
By the early 1990s the software engineering community faced a knowledge transfer problem. Knowledge of how to build computer systems effectively existed, but was confined to a few experts. Even when the knowledge was widely known it was difficult to talk about. The rapid growth of the discipline needed a new way of communicating this knowledge.
In the late 1980s a few engineers in the software development community began to look at Alexander's pattern ideas. Bruce Anderson, Kent Beck, Grady Booch, Jim Coplien, Ward Cunningham, Erich Gamma, Ralph Johnson and others set about applying Alexander's ideas to software. One result of this was the formation of the Hillside Group (www.hillside.net) topromote these patterns. Like Alexander, they used this approach to codify tacit knowledge so that it could be communicated and shared in a wider community.
The patterns identified by Alexander and the software community were not created by them. Patterns can, and do, exist independently of documentation. The world is full of undocumented patterns. Only a few of all possible patterns are documented. To those which are documented, the title pattern is awarded. These patterns codify the events and constructions that happen again and again.
Building on Alexander's ideas, each formally documented pattern contains:
The problem the pattern is addressing
The context in which the pattern is found
The forces that make it hard to solve
How the solution is constructed
The consequences of implementing the solution
Patterns that adopt this approach may be called Alexandrian patterns. There are different formats and styles in which such patterns might be written (see Appendix A) but, in general, as long as they consider these elements, we may call them Alexandrian patterns, or just patterns for short.
Ever since software professionals started writing patterns there have been some patterns which concern themselves more with the business side of the profession than the technical side. For example, the patterns in Organizational Patterns of Agile Software Development (Coplien & Harrison, 2004) are more about organisational form than software development. Fearless Change (Manns & Rising, 2005) originated in the software community, but is really about introducing organisational change.
This book sets out to show how the pattern approach can tackle the major issues of business – strategy – and specifically, business strategy as deployed by companies that create software.
All that jazz
The formation flying metaphor works – especially for engineers – but business people have long p...