PART ONE
PREDICTION ADDICTION
1
FALSE PROFITS
The only function of economic forecasting is to make astrology look respectable.
â JOHN KENNETH GALBRAITH
Who knew what was in the air? Even a breath can be a catalyst.
Enjoying the waters just beyond Narragansett Pier, Rhode Island, the economist Irving Fisher kept swimming. A happy marriage, two daughters. The full professorship at Yale was a lifetime appointment and the future seemed as dazzling as the summer ocean. But looking back to the shore, he was surprised how far the current had carried him. It took all his energy to regain the beach. He arrived at last exhausted, unnerved by the speed with which his glorious future had turned precarious.
For the rest of his life, Fisher would wonder whether that episode in the summer of 1898 had been an early warning sign. Was the swim so tough because he was already infected â or did his exhaustion trigger the crisis? Whichever way it happened, by the autumn Fisher scarcely recognised himself. Everything tired him and every afternoon he ran a fever. His doctor was stumped. Not a man to appreciate uncertainty, Fisher demanded a saliva test. When it came back positive, the physician felt too abashed to face his patient. Instead, he quit, never submitting his bill.
It was left to Fisherâs wife, Margaret, to deliver the diagnosis: tuberculosis. At the time, TB was the single greatest cause of death in the western world. Autopsies showed almost every city dweller to be infected. But even without the data, Fisher knew the danger he faced: as a teenager, his father had died from the disease. Now aged thirty-one, what kind of future did he face?
Millions asked themselves the same question. By 1898, educated people knew that tuberculosis was an airborne bacillus, but there was no vaccine and no certain cure. Nor was there any reliable prognosis: the disease could lie latent for years, even a lifetime, or you could be dead in a matter of weeks. So diagnosis was almost worse than useless. Had Fisher received a life sentence, or just experienced some mild discomfort that would never return? As painful as the disease was the doubt.
As with all epidemics, moralists were quick to construct punitive theories to explain its cause. The disease was divine retribution for alcohol or tobacco consumption, sexual âself-abuseâ, even dancing was suspected. Or perhaps society was to blame: commentators noted that TB thrived in places where urban crowding, pollution, mixed races proliferated. One surgeon, Ambrose Ranney, insisted that whether the disease killed you could be discerned through analysis of the lines of the brow, the hue and texture of the skin.1 From causes to cures, everyone searched for predictive patterns.
Fisher turned to diet. He eschewed meat, forswore alcohol, and became an energetic advocate for prolonged mastication. At Yale, he urged athletes to correlate the length of time that they chewed their food with their athletic performance. He endorsed the new breakfast cereal Grape-Nuts, certain its extreme chewiness would make its consumers stronger. Diet became a life-or-death mission for Fisher â but not only for Fisher. Convinced that the health of a nation determined the wealth of the nation, he estimated the annual economic cost to the US of tuberculosis at $550 million â around $254 billion today.2
Although the TB bacillus had been isolated by Robert Koch in 1882, no cure was known until 1944 when Schatz and Waksman discovered streptomycin. Until then, patients remained suspended in crisis: fearful of the future and desperate for any remedies or signs that might foretell their future. For Irving Fisher, uncertainty was not an abstract idea but a visceral reality.
He was not alone in his uncertainty. Just a casual scan of events at the start of the twentieth century reveals a concatenation of wars, terrorism, political assassinations, earthquakes, royal suicides, epidemics and famine. What did these events portend? Did they spell progress or doom? Market crashes, revolutionary movements, new political parties, scientific breakthroughs, radical technological change and a chaotic cultural scene just barely contained order, anxiety and mayhem. The old-fashioned and the avant-garde â Peter Rabbit, Picasso, Singer Sargent, Munch, Gilbert and Sullivan, Stravinsky, Chekhov and Ibsen â jostled for attention as consumers marvelled at the first plastics, motorbikes, rubber gloves, zippers, telephones, radio programmes, x-rays, colour photographs, cinemas and the Daily Mail. Whole new countries took shape while scientists struggled to understand the impact of four-dimensional geometry, new moons and gases, the new science of relativity and quantum theory.
Whether through fear of the unknown, or hope to capitalise on new trends, a large, eager and susceptible market arose, desperate to know what the future might hold. And for the first time in history, technology provided tools that promised to make forecasting scientific. The telegraph and telephone enabled the collection of large amounts of up-to-date information. The emergence of statistics as a rigorous mathematical discipline, together with the growing sophistication of economics, facilitated serious data analysis. An ever-expanding railway network could disseminate newsletters, newspapers and magazines to an anxious, eager market of punters and pundits.
Astrology became a big commercial business at this time too, but it was in financial markets that forecasting first became a big, important industry. Panics in the US in 1893, 1896, 1901 and 1907 had exposed how little reliable information consumers, investors and managers had about the health of companies, industries or the economy at large. Into that vacuum rushed three men: Irving Fisher, Roger Babson and Warren Persons. All three were eager to sell reassurance, inspiration and advice. Each believed that, through data, they could discern future trends in the markets and hoped to build important businesses doing so. And all three, carrying the diagnosis of TB, viscerally understood the pain of uncertainty and sought to alleviate it.
Almanacs had been around for centuries â supplying farmers with information on sunrises and sunsets, tides and weather â and the new forecasting ventures aspired to something similar for investors: business barometers with which to analyse the present and forecast the future. But the metaphor posed more questions than it answered. With more economic and business data than ever before, how could they tell what was meaningful or trustworthy? It is easy to take the temperature with a thermometer, but in the early days of modern economics, no one quite knew which data was its equivalent. Farmers knew from experience what weather their crops required, but nobody really understood what kinds of economic conditions were needed to temper an overheated market, grow a slow one or stabilise volatility. Did different industries always thrive in the same market conditions? The density of unknowns meant it was left up to the forecasters to choose what mattered to whom, why and when. And they had a field day.
Each of the forecasters built commercial businesses selling their special take on the future. Fisher, whom Milton Friedman considered the greatest economist the United States has ever produced, was one of the first to try to analyse national economies, seen through the lens of the money supply. His working assumption was that too much money in circulation would produce an inflationary boom; too little a recessionary bust. At the time, government didnât measure money supply, which left Fisher trying to do so. He needed indicators of activity â but none existed. He started tracking prices, only to discover that they didnât always move in lock step: some went up as others sank. So he created indexes, aggregates of data he hoped would reveal overall patterns in economic activity. It was impossible to collect everything so he needed to be selective. But how could he identify representative data when he didnât know what the whole contained?
Whatever he chose, Fisherâs theory required mountains of data. He packed his home with employees collecting it on index cards. What his homemade indices revealed was volatility â in prices and in markets. So Fisher became obsessed by a search for stability â where did it come from, what influenced it and what sustained it? How could stable currencies be realised? The more data he collected, the more questions emerged, all needing answers before Fisher could hope to anticipate where the economy was going.
Giddy with new insights, Fisher became one of the worldâs first economic pundits. His Index Number Institute, syndicating indexes and forecasts through newspapers and newsletters, made Fisher famous for financial commentary, analytical nous and his immense capacity for data analysis. Competitive, commercial and publicly spirited, he was easily drawn into commenting on a whole range of topics from prohibition to simplified spelling and calendar reform. But his fortune was made when he sold his card index system to the Rand Kardex Company for $660,000 (approximately $8 million today). Spread thin and often mocked for his humourlessness, Fisher nonetheless commanded attention and credibility for his mathematical rigour that promised to bring economic forecasting one step closer to a science.
Economics has long suffered from âphysics envyâ, and nowhere was that more explicit than in the early days of forecasting. One of Fisherâs rivals, Roger Babson, believed that almost everything in life could be reduced to Newtonâs laws of cause and effect. Like Fisher, Babson had contracted TB as a young man and devised his own eccentric, ascetic health routine involving freezing air and a strict diet. Pictures of him in the Massachusetts winter, dressed in a long woollen gown as he works in front of a wide open window, show a man bent on proving that knowledge and determination could beat any odds. In particular, Babson was on a mission to redress a power imbalance. As a young bond salesman, he had discovered that banks held a monopoly on business information; investors knew only what institutions told them.3 He had seen first-hand the human cost of that exclusivity too: visiting the stock exchange during the panic of 1907, he actually saw men turn grey.4 So Babson brought to his new business an evangelical determination to empower individuals with data as sound and thorough as any bankâs.
Babson wanted people to understand how intricately companies were connected to the economy as a whole, and he became famous for his Babsoncharts â spectacularly baroque graphic designs on which he tried to display the full complexity of an economy: stock and commodity prices, manufacturing data, railroad traffic, agricultural production, building construction, business failures and other indicators of economic output. Onto this data, he placed what he called the ânormal lineâ, indicating periods of expansion and recession. Ever the fervent Newtonian, he put his trust in the Third Law of Motion: for every action, there is an equal and opposite reaction. He believed that a period of depression was always matched by a period of prosperity and that the steeper the decline, the faster the market would recover.5
Like many then and now, Babson imbued economics with morality; cause and effect resonated with crime and punishment. So booms were the product of wasteful exuberance that needed to be purged by sensible self-discipline. These views made him a contrarian: when markets went up, he foresaw extravagance and urged healthy restraint. Profiled as âthe man who refused to dieâ, his success and fame conflated his apparent victory over tuberculosis with the moral lessons implicit in his market predictions â excess spelled danger and health demanded moderation. Each piece of data contained some meaning, he thought, and Babson maintained a boosterish faith that everything â health, behaviour, food, parenting, handwriting â was predictive of something and that he was the man to decode them all.
By 1910, he too had become a national pundit, called on to pronounce on everything from markets to medicine, education, diet and religion. No matter the topic, whenever Babson made predictions, he always looked for excess that needed reining in, or restraint that demanded a bigger push. Both Fisher and Babson became irrepressible entrepreneurs, financing their forecasting businesses with their own money and running them from home. They both worked through a process of deduction, applying their theories to mountains of data in the belief that their efforts would elucidate patterns that predicted the future.
By stark contrast, Warren Persons built his forecasting business, the Harvard Economic Service, right inside the university that funded it, hoping that, far from Wall Street, it could remain aloof from punditry and secure a solid reputation for scholarship and objectivity. A masterful statistician, he took an inductive approach, sceptical that any theories fully captured the complexity of economic markets. The best you could do was watch and measure what was in front of you and ask if you had seen such correlations and patterns before. In essence, he forecasted by analogy, believing that history repeated itself, albeit imperfectly.
The Harvard Economic Service was the worldâs first economic advisory business to serve a worldwide market of the elite, and in the 1920s it began to collaborate with Keynesâ and Beveridgeâs London and Cambridge Economic Service. But for all its academic credentials, a problem lay at the heart of Personsâs approach. Even if he believed himself immune to theory, didnât his attention to some trends over others imply a theory? In denying his assumptions, did he risk being blind to them?
All three men were personally invested in their competing theories and methods â their businesses and professional reputations depended on them, as, to a large degree, did the future of the forecasting industry as a whole. There were fortunes to be made in prophecy and their large, aggressive sales teams competed in a torrid market, each disparaging the others. Persons compared Babsonâs ideas to astrology and said that Fisherâs data was unreliable. Where both menâs methods were opaque and easily castigated as pseudo-science, Persons published hundreds of pages explaining how he worked â which just left him open to intense, methodological criticism from rivals and colleagues alike. Across the industry, rivals sniped at each other, trying to prove that they, and they alone, held the key to the future.
The test came in October 1929. At the beginning of the month, Fisher had been buoyant, claiming that stocks had âreached a permanently high plateauâ. Stability at last! When the market collapsed on 24 October, while conceding there might be a slight price retreat, he saw nothing âin the nature of a crashâ. For months afterwards, he insisted that a rapid recovery was imminent. His faith cost him dear, his son later commenting that âhis eagerness to promote his cause sometimes had a bad influence on his scientific attitude. It distorted his judgement.â6 It also meant that he lost his fortune holding on to his shares in Rand Kardex.
Personsâs Harvard Economic Service was equally blindsided and afterwards maintained that, with no historic precedent for the crash, a normal and swift recovery must follow. Month after month, the service kept predicting a recovery that failed to arrive. Cleaving to his big idea â that the economy always moved in cycles like the tides â proved too rigid in the new environment. The service closed in 1931. That so prestigious a group had failed so dismally soon gave rise to more disturbing questions: did such a commercial venture belong in an academic setting? Had the elite nature of the service and its clients influenced their predictions? Was the Harvard Economic Service unwilling to deliver bad news and, even worse, had its optimism contributed to the crash?
Only Babson, the least scientific of the three men, came out with his reputation enhanced. The reason was simple. Ever the contrarian, he had been predicting a crash every year for the past three years. âI shall repeat what I said at this time last year and the year before; namely that sooner or later a crash is coming . . . Fair weather cannot always continue.â He basked in his triumph, snapping up some of his failing competitors and blaming his rivals for encouraging speculation. But in May 1931, he announced that the market had bottomed out and that it was time to get back into the stock market. He was wrong: the US economy wouldnât recover for a decade.
Persons moved on to consulting, where his statistical punctiliousness was said to dismay clients. Fisher, his reputation in ruins, went bankrupt â his investments had all gone bust. Though Babson was the forecaster who left behind a fortune and the college that still bears his name, his ideas â about markets and medicine â are now discredited. His ânormal lineâ wasnât brilliant, but random. Not one of these three men had accurately foreseen their own legacy. While they had seen themselves as wildly differentiated rivals, in fact these three pioneers had much in common. Beset by personal uncertainty, they believed that deciphering patterns in their data would give them control over their lives. They had more faith than skill, imbuing their theories and data with th...