Emergence

The Connected Lives of Ants, Brains, Cities, and Software

Steven Johnson

• Publisher: Scribner
• Year: 2012
• ISBN:
  9780743218269
• Topic: Ciencias biológicas
• Subtopic: Teoría de sistemas

In the tradition of Being Digital and The Tipping Point, Steven Johnson, acclaimed as a "cultural critic with a poet's heart" (The Village Voice), takes readers on an eye-opening journey through emergence theory and its applications. A NEW YORK TIMES NOTABLE BOOK
A VOICE LITERARY SUPPLEMENT TOP 25 FAVORITE BOOKS OF THE YEAR
AN ESQUIRE MAGAZINE BEST BOOK OF THE YEAR
Explaining why the whole is sometimes smarter than the sum of its parts, Johnson presents surprising examples of feedback, self-organization, and adaptive learning. How does a lively neighborhood evolve out of a disconnected group of shopkeepers, bartenders, and real estate developers? How does a media event take on a life of its own? How will new software programs create an intelligent World Wide Web? In the coming years, the power of self-organization -- coupled with the connective technology of the Internet -- will usher in a revolution every bit as significant as the introduction of electricity. Provocative and engaging, Emergence puts you on the front lines of this exciting upheaval in science and thought.

PART ONE

African anthill (Courtesy of Corbis)

1

The Myth of the Ant Queen

It’s early fall in Palo Alto, and Deborah Gordon and I are sitting in her office in Stanford’s Gilbert Biological Sciences building, where she spends three-quarters of the year studying behavioral ecology. The other quarter is spent doing fieldwork with the native harvester ants of the American Southwest, and when we meet, her face still retains the hint of a tan from her last excursion to the Arizona desert.

I’ve come here to learn more about the collective intelligence of ant colonies. Gordon, dressed neatly in a white shirt, cheerfully entertains a few borderline-philosophical questions on group behavior and complex systems, but I can tell she’s hankering to start with a hands-on display. After a few minutes of casual rumination, she bolts up out of her chair. “Why don’t we start with me showing you the ants that we have here,” she says. “And then we can talk about what it all means.”

She ushers me into a sepulchral room across the hallway, where three long tables are lined up side by side. The initial impression is that of an underpopulated and sterilized pool hall, until I get close enough to one of the tables to make out the miniature civilization that lives within each of them. Closer to a Habitrail than your traditional idea of an ant farm, Gordon’s contraptions house an intricate network of plastic tubes connecting a dozen or so plastic boxes, each lined with moist plaster and coated with a thin layer of dirt.

“We cover the nests with red plastic because some species of ants don’t see red light,” Gordon explains. “That seems to be true of this species too.” For a second, I’m not sure what she means by “this species”—and then my eyes adjust to the scene, and I realize with a start that the dirt coating the plastic boxes is, in fact, thousands of harvester ants, crammed so tightly into their quarters that I had originally mistaken them for an undifferentiated mass. A second later, I can see that the whole simulated colony is wonderfully alive, the clusters of ants pulsing steadily with movement. The tubing and cramped conditions and surging crowds bring one thought immediately to mind: the New York subway system, rush hour.

At the heart of Gordon’s work is a mystery about how ant colonies develop, a mystery that has implications extending far beyond the parched earth of the Arizona desert to our cities, our brains, our immune systems—and increasingly, our technology. Gordon’s work focuses on the connection between the microbehavior of individual ants and the overall behavior of the colonies themselves, and part of that research involves tracking the life cycles of individual colonies, following them year after year as they scour the desert floor for food, competing with other colonies for territory, and—once a year—mating with them. She is a student, in other words, of a particular kind of emergent, self-organizing system.

Dig up a colony of native harvester ants and you’ll almost invariably find that the queen is missing. To track down the colony’s matriarch, you need to examine the bottom of the hole you’ve just dug to excavate the colony: you’ll find a narrow, almost invisible passageway that leads another two feet underground, to a tiny vestibule burrowed out of the earth. There you will find the queen. She will have been secreted there by a handful of ladies-in-waiting at the first sign of disturbance. That passageway, in other words, is an emergency escape hatch, not unlike a fallout shelter buried deep below the West Wing.

But despite the Secret Service–like behavior, and the regal nomenclature, there’s nothing hierarchical about the way an ant colony does its thinking. “Although queen is a term that reminds us of human political systems,” Gordon explains, “the queen is not an authority figure. She lays eggs and is fed and cared for by the workers. She does not decide which worker does what. In a harvester ant colony, many feet of intricate tunnels and chambers and thousands of ants separate the queen, surrounded by interior workers, from the ants working outside the nest and using only the chambers near the surface. It would be physically impossible for the queen to direct every worker’s decision about which task to perform and when.” The harvester ants that carry the queen off to her escape hatch do so not because they’ve been ordered to by their leader; they do it because the queen ant is responsible for giving birth to all the members of the colony, and so it’s in the colony’s best interest—and the colony’s gene pool—to keep the queen safe. Their genes instruct them to protect their mother, the same way their genes instruct them to forage for food. In other words, the matriarch doesn’t train her servants to protect her, evolution does.

Popular culture trades in Stalinist ant stereotypes—witness the authoritarian colony regime in the animated film Antz—but in fact, colonies are the exact opposite of command economies. While they are capable of remarkably coordinated feats of task allocation, there are no Five-Year Plans in the ant kingdom. The colonies that Gordon studies display some of nature’s most mesmerizing decentralized behavior: intelligence and personality and learning that emerges from the bottom up.

I’m still gazing into the latticework of plastic tubing when Gordon directs my attention to the two expansive white boards attached to the main colony space, one stacked on top of the other and connected by a ramp. (Imagine a two-story parking garage built next to a subway stop.) A handful of ants meander across each plank, some porting crumblike objects on their back, others apparently just out for a stroll. If this is the Central Park of Gordon’s ant metropolis, I think, it must be a workday.

Gordon gestures to the near corner of the top board, four inches from the ramp to the lower level, where a pile of strangely textured dust—littered with tiny shells and husks—presses neatly against the wall. “That’s the midden,” she says. “It’s the town garbage dump.” She points to three ants marching up the ramp, each barely visible beneath a comically oversize shell. “These ants are on midden duty: they take the trash that’s left over from the food they’ve collected—in this case, the seeds from stalk grass—and deposit it in the midden pile.”

Gordon takes two quick steps down to the other side of the table, at the far end away from the ramp. She points to what looks like another pile of dust. “And this is the cemetery.” I look again, startled. She’s right: hundreds of ant carcasses are piled atop one another, all carefully wedged against the table’s corner. It looks brutal, and yet also strangely methodical.

I know enough about colony behavior to nod in amazement. “So they’ve somehow collectively decided to utilize these two areas as trash heap and cemetery,” I say. No individual ant defined those areas, no central planner zoned one area for trash, the other for the dead. “It just sort of happened, right?”

Gordon smiles, and it’s clear that I’ve missed something. “It’s better than that,” she says. “Look at what actually happened here: they’ve built the cemetery at exactly the point that’s furthest away from the colony. And the midden is even more interesting: they’ve put it at precisely the point that maximizes its distance from both the colony and the cemetery. It’s like there’s a rule they’re following: put the dead ants as far away as possible, and put the midden as far away as possible without putting it near the dead ants.”

I have to take a few seconds to do the geometry myself, and sure enough, the ants have got it right. I find myself laughing out loud at the thought: it’s as though they’ve solved one of those spatial math tests that appear on standardized tests, conjuring up a solution that’s perfectly tailored to their environment, a solution that might easily stump an eight-year-old human. The question is, who’s doing the conjuring?

It’s a question with a long and august history, one that is scarcely limited to the collective behavior of ant colonies. We know the answer now because we have developed powerful tools for thinking about—and modeling—the emergent intelligence of self-organizing systems, but that answer was not always so clear. We know now that systems like ant colonies don’t have real leaders, that the very idea of an ant “queen” is misleading. But the desire to find pacemakers in such systems has always been powerful—in both the group behavior of the social insects, and in the collective human behavior that creates a living city.

* * *

Records exist of a Roman fort dating back to A.D. 76 situated at the confluence of the Medlock and Irwell Rivers, on the northwestern edge of modern England, about 150 miles from London. Settlements persisted there for three centuries, before dying out with the rest of the empire around A.D. 400. Historians believe that the site was unoccupied for half a millennium, until a town called Manchester began to take shape there, the name derived from the Roman settlement Mamucium—Latin for “place of the breastlike hill.”

Manchester subsisted through most of the millennium as a nondescript northern-England borough: granted a charter in 1301, the town established a college in the early 1400s, but remained secondary to the neighboring town of Salford for hundreds of years. In the 1600s, the Manchester region became a node for the wool trade, its merchants shipping goods to the Continent via the great ports of London. It was impossible to see it at the time, but Manchester—and indeed the entire Lancashire region—had planted itself at the very center of a technological and commercial revolution that would irrevocably alter the future of the planet. Manchester lay at the confluence of several world-historical rivers: the nascent industrial technologies of steam-powered looms; the banking system of commercial London; the global markets and labor pools of the British Empire. The story of that convergence has been told many times, and the debate over its consequences continues to this day. But beyond the epic effects that it had on the global economy, the industrial takeoff that occurred in Manchester between 1700 and 1850 also created a new kind of city, one that literally exploded into existence.

The statistics on population growth alone capture the force of that explosion: a 1773 estimate had 24,000 people living in Manchester; the first official census in 1801 found 70,000. By the midpoint of the century, there were more than 250,000 people in the city proper—a tenfold increase in only seventy-five years. That growth rate was as unprecedented and as violent as the steam engines themselves. In a real sense, the city grew too fast for the authorities to keep up with it. For five hundred years, Manchester had technically been considered a “manor,” which meant, in the eyes of the law, it was run like a feudal estate, with no local government to speak of—no city planners, police, or public health authorities. Manchester didn’t even send representatives to Parliament until 1832, and it wasn’t incorporated for another six years. By the early 1840s, the newly formed borough council finally began to institute public health reforms and urban planning, but the British government didn’t officially recognize Manchester as a city until 1853. This constitutes one of the great ironies of the industrial revolution, and it captures just how dramatic the rate of change really was: the city that most defined the future of urban life for the first half of the nineteenth century didn’t legally become a city until the great explosion had run its course.

The result of that discontinuity was arguably the least planned and most chaotic city in the six-thousand-year history of urban settlements. Noisy, polluted, massively overcrowded, Manchester attracted a steady stream of intellectuals and public figures in the 1830s, traveling north to the industrial magnet in search of the modern world’s future. One by one, they returned with stories of abject squalor and sensory overload, their words straining to convey the immensity and uniqueness of the experience. “What I have seen has disgusted and astonished me beyond all measure,” Dickens wrote after a visit in the fall of 1838. “I mean to strike the heaviest blow in my power for these unfortunate creatures.” Appointed to command the northern districts in the late 1830s, Major General Charles James Napier wrote: “Manchester is the chimney of the world. Rich rascals, poor rogues, drunken ragamuffins and prostitutes form the moral. . . . What a place! The entrance to hell, realized.” De Toqueville visited Lancashire in 1835 and described the landscape in language that would be echoed throughout the next two centuries: “From this foul drain the greatest stream of human industry flows out to fertilize the whole world. From this filthy sewer pure gold flows. Here humanity attains its most complete development and its most brutish; here civilization works its miracles, and civilized man is turned back almost into a savage.”

But Manchester’s most celebrated and influential documentarian was a young man named Friedrich Engels, who arrived in 1842 to help oversee the family cotton plant there, and to witness firsthand the engines of history bringing the working class closer to self-awareness. While Engels was very much on the payroll of his father’s firm, Ermen and Engels, by the time he arrived in Manchester he was also under the sway of the radical politics associated with the Young Hegelian school. He had befriended Karl Marx a few years before and had been encouraged to visit Manchester by the socialist Moses Hess, whom he’d met in early 1842. His three years in England were thus a kind of scouting mission for the revolution, financed by the capitalist class. The book that Engels eventually wrote, The Condition of the Working Class in England, remains to this day one of the classic tracts of urban history and stands as the definitive account of nineteenth-century Manchester life in all its tumult and dynamism. Dickens, Carlyle, and Disraeli had all attempted to capture Manchester in its epic wildness, but their efforts were outpaced by a twenty-four-year-old from Prussia.

But The Condition is not, as might be expected, purely a document of Manchester’s industrial chaos, a story of all that is solid melting into air, to borrow a phrase Engels’s comrade would write several years later. In the midst of the city’s insanity, Engels’s eye is drawn to a strange kind of order, in a wonderful passage where he leads the reader on a walking tour of the industrial capital, a tour that reveals a kind of politics built into the very topography of the city’s streets. It captures Engels’s acute powers of observation, but I quote from it at length because it captures something else as well—how difficult it is to think in models of self-organization, to imagine a world without pacemakers.

The town itself is peculiarly built, so that someone can live in it for years and travel into it and out of it daily without ever coming into contact with a working-class quarter or even with workers—so long, that is to say, as one confines himself to his business affairs or to strolling about for pleasure. This comes about mainly in the circumstances that through an unconscious, tacit agreement as much as through conscious, explicit intention, the working-class districts are most sharply separated from the parts of the city reserved for the middle class. . . .

I know perfectly well that this deceitful manner of building is more or less common to all big cities. I know as well that shopkeepers must in the nature of the business take premises on the main thoroughfares. I know in such streets there are more good houses than bad ones, and that the value of land is higher in their immediate vicinity than in neighborhoods that lie at a distance from them. But at the same time I have never come across so systematic a seclusion of the working class from the main streets as in Manchester. I have never elsewhere seen a concealment of such fine sensibility of everything that might offend the eyes and nerves of the middle classes. And yet it is precisely Manchester that has been built less according to a plan and less within the limitations of official regulations—and indeed more through accident—than any other town. Still . . . I cannot help feeling that the liberal industrialists, the Manchester “bigwigs,” are not so altogether innocent of this bashful style of building.

You can almost hear the contradictions thundering against each other in this passage, like the “dark satanic mills” of Manchester itself. The city has built a cordon sanitaire to separate the industrialists from the squalor they have unleashed on the world, concealing the demoralization of Manchester’s working-class districts—and yet that disappearing act comes into the world without “conscious, explicit intention.” The city seems artfully planned to hide its atrocities, and yet it “has been built less according to a plan” than any city in history. As Steven Marcus puts it, in his history of the young Engels’s sojourn in Manchester, “The point to be taken is that this astonishing and outrageous arrangement cannot fully be understood as the result of a plot, or even a deliberate design, although those in whose interests it works also control it. It is indeed too huge and too complex a state of organized affairs ever to have been thought up in advance, to have preexisted as an idea.”

Those broad, glittering avenues, in other words, suggest a Potemkin village without a Potemkin. That mix of order and anarchy is what we now call emergent behavior. Urban critics since Lewis Mumford and Jane Jacobs have known that cities have lives of their own, with neighborhoods clustering into place without any Robert Moses figure dictating the plan from above. But that understanding has entered the intellectual mainstream only in recent years—when Engels paced those Manchester streets in the 1840s, he was left groping blindly, trying to find a culprit for the city’s fiendish organization, even as he acknowledged that the city was notoriously unplanned. Like most intellectual histories, the development of that new understanding—the sciences of complexity and self-organization—is a complicated, multithreaded tale, with many agents interacting over its duration. It i...