The Social Labs Revolution

A New Approach to Solving our Most Complex Challenges

Zaid Hassan

• Publisher: Berrett-Koehler Publishers
• Year: 2014
• ISBN:
  9781626560758
• Edition: 1
• Topic: Politik & Internationale Beziehungen
• Subtopic: Sozialpolitik

Current responses to our most pressing societal challenges—from poverty to ethnic conflict to climate change—are not working. These problems are incredibly dynamic and complex, involving an ever-shifting array of factors, actors, and circumstances. They demand a highly fluid and adaptive approach, yet we address them by devising fixed, long-term plans. Social labs, says Zaid Hassan, are a dramatically more effective response. Social labs bring together a diverse a group of stakeholders—not to create yet another five-year plan but to develop a portfolio of prototype solutions, test those solutions in the real world, use the data to further refine them, and test them again. Hassan builds on a decade of experience—as well as drawing from cutting-edge research in complexity science, networking theory, and sociology—to explain the core principles and daily functioning of social labs, using examples of pioneering labs from around the world. He offers a new generation of problem solvers an effective, practical, and exciting new vision and guide.

1
The Perfect Storm of Complexity

When you want to know how things really work, study them when they’re coming apart.

—William Gibson, Zero History

For every complex problem there is an answer that is clear, simple, and wrong.

— H. L. Mencken

Humanity has always suffered plagues, famines, floods, and warfare. In modern times we have faced new horrors, such as nuclear weapons and AIDS. One common stance toward our current challenges is that we will adapt just as we have always adapted. The trouble with this stance is that our current challenges are profoundly different from those of the past. Our familiar modern responses no longer work because they’re based on a fundamental misunderstanding of what we are facing.

THE PERFECT CHALLENGE

Just how different our challenges are crystallized for me in the summer of 2008. It began with a mysterious call from two strangers. I met them in an empty cafe on Cowley Road in Oxford, not far from where I live. Both had been working in Yemen for a number of years. They wanted to know if we could help. I knew very little about Yemen and so asked them to explain the situation to me. The pair, Henry Thompson and Ginny Hill, spoke in hushed voices, occasionally looking around to make sure no one else was listening. I was bemused at their behavior and not quite sure what to make of them.

Yemen, they told me, was in serious trouble. It was collapsing. The facts were startling. Bordering Saudi Arabia and Oman, on the other side of the Red Sea from Somalia, Yemen occupied a geostrategic location due to the Suez Canal and its proximity to the oil fields of Saudi Arabia and the United Arab Emirates. One of the oldest civilizations in the Middle East, it also had the youngest and fastest growing population in the region, over twenty-three million people, 50 percent of whom were under fifteen.¹

First, Al Quaeda was using Yemen as a major base for operations. Second, the country was suffering from two incipient civil wars, which threatened to flare up at any moment. One was a secessionist movement in the south, and the other involved a religious minority in the north. In addition, Yemenis were running out of what meager resources they had: water, oil, food, and foreign exchange to buy food. Yet, Yemen had four times as many AK47s as people.²

Finally, they explained, the crux of the problem was that a cabal of criminals and quasi-criminals ran the country, a situation sometimes known as state-capture. This shadow elite lived behind anti-missile walls and in some cases held no official positions despite wielding great influence. When I asked about official channels, they looked at each other and shrugged. Could we help?

WHAT IS A COMPLEX SOCIAL CHALLENGE?

The situation in Yemen is a textbook example of a complex social challenge because of three characteristics: (1) the situation is emergent, (2), as a result, there is a constant flow of information to negotiate, and (3) this means actors are constantly adapting their behavior.³

Complex social challenges are emergent because their properties arise from the interaction of many parts. Imagine the difference between throwing a rock and throwing a live bird. The rock will follow a path that is predictable, that is, it can be predicted with a high degree of accuracy in advance. The path of the bird, on the other hand, is emergent, which means that path cannot be predicted in advance. It emerges from the interactions of many factors from the physiology of the bird to environmental factors. The system of the person (throwing the bird) and the bird is therefore said to be characterized by emergence.

In complex systems new information is constantly being generated.⁴ When we study a complex system, we are deluged by new information. If we tied a GPS to the bird and tracked its movements, we would be capturing a new stream of information about where the bird was going. (According to Nate Silver, “IBM estimates we are generating 2.5 quintillion bytes of data per day, more than 90 percent of which was created in the last two years.”⁵)

This new information gives rise to the third characteristic of a complex system, that of adaptive behavior. This means that actors in complex systems are constantly and autonomously adjusting their behaviors in response to new information. This feedback loop in turn gives rise to a whole new set of emergent characteristics. If our task is to re-capture the bird once it’s been thrown, then we use information to adapt our behaviors to ensure we succeed.

These three characteristics make complex challenges distinct by nature from technical challenges. Ronald Heifitz and his colleagues at Harvard’s Kennedy School of Government define a challenge as being technical when the problem and the solution are clearly defined.⁶ And they point out that confusing adaptive, or complex, challenges with technical challenges is a classic error.

An example of a technical challenge is sending a man to the moon. The problem is clearly defined and the solution unequivocal. Implementation may require solving many difficult problems, but the desired outcome is plainly understood and agreed upon. In contrast, multiple perceptions of both the problem and the solution are characteristic of complex systems.

Complex challenges are therefore dynamic and can change in unexpected ways over time, whereas technical challenges are relatively stable and static in comparison. The nature of gravity, for example, is not changing while we try to come up with solutions for putting a man on the moon. This is just one reason why it is hard to address complex social challenges.

In the past, everything was less connected. Today, interconnectivity is rapidly increasing, creating an age defined by its complexity. This connectivity has many dividends, but it also means that our landscape of challenges has changed dramatically in the last few decades. In the past, problems could be dealt with in isolation, while today, most of our most intractable social challenges are deeply interconnected. They don’t respect man-made boundaries, such as national borders. The nature of interconnectivity means that we are seeing challenges that are entirely new and fast changing.

These challenges are sometimes referred to as wicked problems, a phrase coined in the early 1970s.⁷ The trouble with the word wicked is that it makes us think that complex situations are somehow deviations from a non-wicked norm, that they are somehow temporary aberrations. And the problem, if you like, with the word problem is that it conveys the impression that everyone thinks of the situation as a problem (when some actors, typically those holding minority positions, might not).

One practitioner compares christening complex challenges as wicked to a story of a grandfather and the coming of cars.⁸ The grandfather couldn’t understand why cars didn’t behave like horses (resulting in many accidents) and considered them wicked. Much as we might love our grandfathers, calling complex social challenges wicked betrays a way of thinking that doesn’t make much sense today. Forty years ago we had just started to wrap our heads around the idea of complexity. Since then we have learned a lot, and many ideas from complexity science are in common use. Complexity is the norm for us—not an anomaly—and there is no returning to a simpler “non-wicked” time.

THE FUTILE OPTIMISM OF OPTIMIZATION

It’s 1959. The USSR is on the brink of Utopia. Comrades, let’s optimize!

—Francis Spufford, Red Plenty

Today it is common to address a wide range of complex social challenges using methods that are technical and planning based. Together they define a culturally dominant technocratic approach, which characterizes efforts at addressing challenges as diverse as public health care, environmental degradation, poverty, and inequality.

This dominant technocratic approach was born during the early twentieth century, a time when the belief that science would solve all human problems was widespread. The work of mathematicians such as Kurt Gödel and physicists such as Werner Heisenberg shattered this belief. By then, however, the technocratic paradigm had rooted itself deeply in an entire generation of problem solvers, who then passed it on.

Technocratic approaches typically seek to optimize, that is, to incrementally improve a situation through efficiency gains. For example, if ten thousand people are hungry, then a technocratic approach would seek to ensure that every day some of these people were fed, thus incrementally improving the situation. The end goal, of course, is to ensure that all ten thousand people are fed. This is a classic optimization strategy.

Optimization makes sense in some instances, such as when the number of hungry people is static and not increasing. Economists call this inelastic demand, as opposed to elastic demand.⁹ This means that if we manage to feed two hundred hungry people per day, in fifty days we would have fed all ten thousand people, therefore optimizing our way toward solving the problem of ten thousand hungry people.

This strategy is dramatically less effective in dynamic situations. Imagine that we feed ten thousand hungry people at a rate of two hundred per day. If, for whatever reason, the number of hungry people increases by 5 percent per day (compounded), then we’re in trouble. After five days of feeding two hundred people a day, we end up with just under 11,300 hungry people. After 10 days we end up with just over 13,300 hungry people, after 50 days we wind up with nearly 70,000, and so on. The dream of optimization, of course, is the other way round—that we increase the number of hungry people we feed every day by a percentage, which, when compounded over years, leads to a utopian society free of hunger.¹⁰ Of course, all of this assumes that gains will not be wiped out by unexpected events, such as a famine or some other natural disaster.

Another problem with technocratic approaches, including optimization, is that it addresses parts in isolation, rather than the whole. This could look like feeding a small number of hungry people by cutting down massive swathes of rainforest, which helps a small minority, while vast resources are spent with massive longterm negative impact. A side effect of optimization is that the underlying causal dynamics are frequently untouched.

This is what’s happening in Yemen with malnutrition. The system is generating more malnourished people every day than can be fed. Efforts to support them are helping small parts and are being outstripped by the dynamic nature of the challenge, where the problem as a whole is getting worse day by day.

The same logic applies to many issues, including climate change, deforestation, and poverty. Imagine that ten new light bulbs are turned on every second, each emitting a tiny puff of carbon dioxide. This pumps greenhouse gases into the atmosphere, which increases the risk of dangerous climate change.

An optimization response would be to turn off three light bulbs every second, striving in time to turn off four or five, and believe this is adequate. Unfortunately this leaves us with a net increase of emissions, despite our efforts. That is what’s happening with greenhouse gases dramatically increasing the probability of dangerous climate change.

All complex challenges have what could be thought of as an engine that produces the symptoms we are most concerned about, be that too many hungry people or too many greenhouse gas emissions. We see these symptoms as trends. For example, one of the trends governing the situation in Yemen is population growth, which, in itself, is not a problem. But when coupled with other trends, such as steadily declining agricultural productivity, we can see how it creates the complex social challenge of malnutrition.

This reflects a situation where demand for different forms of capital is increasing, including natural capital, such as fossil fuels and food. Simultaneously, there is a decline in our ability to meet this increasing demand sustainably. This is represented, for example, by declining forests, topsoil loss, less fresh water, and the shrinking envelope of carbon dioxide we can safely emit—which puts limits on how much fossil fuel we can safely burn. In other words, we are now hitting boundaries beyond which our actions seem to be causing irreparable damage to critical ecosystems.

It’s not simply that we’re running out of resources. The story is more complex. Ramez Naam demonstrates how we have used technological innovation to produce greater output from the same natural resources. For example, we have managed to dramatically increase yields from the same acre of land and convert greater percentages of solar energy into electricity. While the efficiencies are getting better and costs are dropping, they are not dropping fast enough to shift the underlying negative trends. Furthermore, market-based approaches have yet to figure out what to do with the environmental consequences of economic growth.¹¹

Technocratic approaches, therefore, represent a bet, a “grand wager” that our ability to optimize will be faster than the rate at which our problems grow.¹² If our problems are growing exponentially and our ability to optimize is growing linearly (or worse, declining), then we are staring at a mathematical certainty of collapse. This is what happened with the Soviet Union and what’s currently happening with many responses to complex social challenges across the world.

YEMEN AS A NATURAL EXPERIMENT

My first response to the request for helping in Yemen was “No, of course we can’t help.” The situation was too far along in its trajectory of collapse. Henry and Ginny wanted to bring the elite—including the shadow elite—into a room and run a scenario planning exercise on the future of Yemen.¹³ The elite would then see the implications of what they were doing to the country, and this insight would cause them to act proactively in the interests of the whole.

I pointed out that the shadow elite would not voluntarily step forward into such a conversation. Our usual approaches would not work with people who were loath to step up in any formal way, which is what defines a shadow elite.

Originally I assumed this situation was unique to Yemen. However, I later came across the work of Janine Wedel, a professor and author, who argues that the phenomenon of the shadow elite is widespread: “A new breed of players has arisen in the past several decades … whose manoeuvrings are beyond the traditional mechanisms of accountability. They, for example, play multiple, overlapping, and not fully disclose...