Trading at the Speed of Light

How Ultrafast Algorithms Are Transforming Financial Markets

Donald MacKenzie

• Publisher: Princeton University Press
• Year: 2021
• ISBN:
  9780691217796
• Topic: Business
• Subtopic: Financial Engineering
• Index: Business

A remarkable look at how the growth, technology, and politics of high-frequency trading have altered global financial markets In today's financial markets, trading floors on which brokers buy and sell shares face-to-face have increasingly been replaced by lightning-fast electronic systems that use algorithms to execute astounding volumes of transactions. Trading at the Speed of Light tells the story of this epic transformation. Donald MacKenzie shows how in the 1990s, in what were then the disreputable margins of the US financial system, a new approach to trading—automated high-frequency trading or HFT—began and then spread throughout the world. HFT has brought new efficiency to global trading, but has also created an unrelenting race for speed, leading to a systematic, subterranean battle among HFT algorithms.In HFT, time is measured in nanoseconds (billionths of a second), and in a nanosecond the fastest possible signal—light in a vacuum—can travel only thirty centimeters, or roughly a foot. That makes HFT exquisitely sensitive to the length and transmission capacity of the cables connecting computer servers to the exchanges' systems and to the location of the microwave towers that carry signals between computer datacenters. Drawing from more than 300 interviews with high-frequency traders, the people who supply them with technological and communication capabilities, exchange staff, regulators, and many others, MacKenzie reveals the extraordinary efforts expended to speed up every aspect of trading. He looks at how in some markets big banks have fought off the challenge from HFT firms, and how exchanges sometimes engineer technical systems to favor certain types of algorithms over others.Focusing on the material, political, and economic characteristics of high-frequency trading, Trading at the Speed of Light offers a unique glimpse into its influence on global finance and where it could lead us in the future.

1

Introduction

Walk down Broad Street toward the southern tip of Manhattan, and you pass the imposing neoclassical façade of the New York Stock Exchange, police barriers, and—in normal times—tourists taking photographs. Throughout the twentieth century, that famous building, crammed with human traders, epitomized what “finance” meant. A couple of minutes’ walk farther south, you would most likely pass 50 Broad Street without a second glance. It has a handsome frontage, and has been renovated internally, but is otherwise an ordinary Manhattan office building (see figure 1.1). In 1993, that stretch of Broad Street, then scruffy and neglected, struck a New York Times journalist as exemplifying downtown’s decline.¹ More than in any other single place, though, what happened at 50 Broad Street in the 1990s and early 2000s transformed the world’s financial markets. Now, just one trace of that role remains: inscribed in panels attached to the stonework above a storefront (which, despite the area’s revival, has been empty for years) is the word “island.”²

Island, launched in 1996, was an electronic venue for the trading of US shares. It was not the first such venue, but none of its predecessors had changed the financial system radically. Some had gone out of business; some had been assimilated into existing ways of doing things; some had succeeded modestly but had not come to occupy central roles. Island was different. Its computer system, packed into the basement of 50 Broad Street, consisted almost entirely of cheap machines of the kind you could have bought in a computer store, but it was blazingly fast by the standards of the 1990s. The interviewee I am calling AF told me that if Island’s system received both a bid to buy shares and an offer to sell the same shares at the same price, it could execute a trade in a couple of milliseconds (thousandths of a second), a thousand times faster than the more mainstream electronic system to which it was most comparable, Instinet. To human eyes, trading on Island appeared instantaneous.

FIGURE 1.1. 50 Broad Street. Author’s photograph.

Just as consequential as Island’s speed was that machines started to trade on it. There had been previous efforts to automate trading, but often they had not gone smoothly. It could be difficult for an automated trading system to interact seamlessly with exchanges’ systems, which in the 1980s and 1990s were usually designed on the assumption that traders were human beings, not machines. Indeed, those who ran exchanges’ early electronic trading systems often protected their human users from “unfair” automated competition by prohibiting the direct connection of computers to them. In the privacy of their offices, traders found ways to circumvent the prohibition—sometimes even constructing robotic devices to hit the keys of terminals designed for human users (one such device is shown in figure 1.2)—but doing this was cumbersome.³ Island, in contrast, was machine friendly from the outset. At its core was a set of “order books”: electronic files, one for each stock, of the bids to buy the shares in question and of the offers to sell them. Every time Island’s computer system executed a trade or received a bid, an offer, or a cancellation of an order, an electronic message was sent out via a continuous datafeed that traders’ computers could use to maintain an up-to-date electronic mirror of Island’s order books. It was also straightforward for those computers to send Island bids and offers in a fast, succinct, standardized electronic format.

FIGURE 1.2. Lehman Brothers “Clackatron” (ca. 2002), used to strike the keys of an EBS (Electronic Broking Services) foreign-exchange trading keypad. Photograph courtesy of interviewee FL.

As the machines that traded on Island got faster, the delays that were inevitable if their orders needed to be transmitted to lower Manhattan through hundreds of miles of fiber-optic cable became ever more salient. Dave Cummings, founder of the Kansas City high-frequency trading firm Tradebot (“Trading Robot”), told the Wall Street Journal in 2006 that he had come to realize that the 10 milliseconds it took a signal to get from Kansas City to 50 Broad Street put his firm at a disadvantage: “We were excluded because of the speed of light” (Lucchetti 2006). Starting around 2002, the firms whose machines traded on Island began to move them into 50 Broad Street, at first informally (a web-services firm that had offices in the building hosted their computer servers) and then—in a formal, paid-for arrangement with Island—placing them in Island’s computer room in the building’s basement, next to Island’s heart, the “matching engine”: the system that managed its order books and executed trades.

What emerged in and around 50 Broad Street (“emerged” is the right word: no one planned it) is this book’s topic: high-frequency trading, or HFT. The practice emerged before the name did; as far as I can tell, the term first came into use at the Chicago hedge fund Citadel in the early 2000s. HFT is “proprietary” automated trading that takes place at speeds far faster than an unaided human can trade and in which trading’s profitability is inherently dependent on its speed.⁴ (The goal of proprietary trading is direct trading profit, rather than, for example, earning fees by executing trades on behalf of others.) Although the human beings employed by HFT firms to design and supervise trading algorithms often refer to themselves as traders, the trading itself is actually done by those computer algorithms. Humans write the algorithms and (less often now than in HFT’s early years) sometimes tweak their parameters during the trading day, but the decisions to place bids to buy and offers to sell are made by the algorithm, not the human being.

HFT algorithms trade both with each other and with other categories of algorithm, such as the “execution algorithms” used by institutional investors—and by banks or other brokers acting on behalf of these investors—to break up a large order to buy or sell shares (or other financial instruments) into much smaller, low-profile “child” orders.⁵ HFT firms’ algorithms also interact with orders placed manually by human beings, for example by those whom market participants refer to as “retail” (individual investors). Only a minority of retail orders, though, end up being traded on exchanges such as the New York Stock Exchange. Most are executed directly by what are sometimes called wholesalers (which are often branches of HFT firms), who pay the brokers via whom retail investors trade to send them these orders.⁶

HFT firms, in aggregate, trade on a giant scale. For example, as we will see in chapter 4, in just over two months in 2015, eight HFT firms traded Treasurys worth in total about $7 trillion. (Treasurys are the sovereign debt securities of the United States. A trillion is a million million.) The anonymity of most of today’s trading makes it difficult in most cases to be certain just how much of it is HFT, but observers often estimate that HFT accounts for around half of all trading on many of the world’s most important markets (see, e.g., Meyer and Bullock 2017; Meyer, Bullock, and Rennison 2018).

The HFT firms that are responsible for these huge volumes of trading are typically recently established and small. Only a small number date from before 2000, and even an HFT firm with no more than a few dozen employees can be a significant player. Consider, for example, Virtu, an HFT firm whose headquarters, as it happens, are just a few blocks away from 50 Broad Street. Virtu’s primary activity is “market-making”—continuously posting both bids to buy shares or other financial instruments and slightly higher-priced offers to sell them—and it does this in more than 25,000 different instruments traded in 36 countries. It is responsible, for example, for around a fifth of all US share trading.⁷ It rose to its dominant position, my interviewees report, while employing no more than 150 people (its headcount has risen recently because of its acquisition of two firms with more labor-intensive businesses).⁸

In particular niches, even firms with only a handful of employees can be important. In 2019, an interviewee calmly told me that his tiny European HFT firm was responsible for 5 percent of all the share trading in India. Some big banks used to be active in HFT, but their efforts were often less than fully successful; the rapid development of the fast, highly specialized software systems that are needed can be difficult in a large, bureaucratic organization. Banks are still engaged in market-making in some classes of financial instrument (such as those discussed in chapter 4: foreign exchange and governments’ sovereign bonds), albeit often using systems that are slow by HFT standards, but large-scale use of other HFT strategies by banks was effectively ended by the curbs on banks’ proprietary trading that followed the 2008 banking crisis.

The HFT firms I have visited differ widely. Some had offices in unremarkable or even scruffy buildings; others had spectacular views over Lake Michigan, Manhattan, or Greater London. The décor is generally bland, although as I sat waiting for an interviewee in one HFT firm’s new offices, some of the owner’s art collection was ready to be hung. The paintings were wrapped and unlabeled, but I’m told they are very fine: the owner has good taste and the firm has been highly successful. More often, though, HFT firms’ premises could pass for those of a generic dot-com firm, and they usually have something of the relaxed feel of a software start-up. The employees of HFT firms are mostly young and—at least in the roles closest to trading—mostly male. Office kitchens, for example, often contain multiple boxes of breakfast cereal, stereotypically young men’s food. I am happy to report, though, that the sexist pinups that sometimes used to disfigure trading floors are no longer to be seen. Almost no one in HFT routinely wears a business suit—it is common for me, as the visitor, to be the only person wearing a tie, and I’ve been told off for being overdressed—and the shouting and swearing that used to be heard on banks’ trading floors is less common in HFT firms. That might, of course, be because of my presence, but interviewees tell me that such behavior is indeed less prevalent. As discussed below, I have visited firms only in the US and Europe. There, at least, white faces predominate, though often intermingled with those of South Asian or Chinese extraction, while African Americans, for example, seem rarer.

The internal organization of the HFT firms from which my interviewees come varies. Some operate as unified entities, without even the traditional individual P&L (a trader’s profit or loss, the prime determinant of her/his bonus); one firm had a computerized “signal library”—an electronic compendium of data patterns useful to HFT algorithms—that was accessible to all its traders and software developers. Just as Lange (2016) discovered, though, other HFT firms are divided into strictly separate trading teams, with deliberate barriers to communication. One firm, for example, physically separates teams by placing a row of administrative staff between them, and in its main offices even plays white noise between the rows to reduce the chance of members of one team overhearing what is said by members of another. Another firm compartmentalizes its trading by dividing up its long, narrow trading room with white curtains that prevent members of one team from seeing what others are doing. At one compartmentalized firm, said a young trader (interviewee AC) who worked there, “you … could get in trouble for being in the next room talking to someone you’re not supposed to talk to.”⁹

High-frequency trading, however, does not actually happen in these rooms. Instead, it takes place in the computer datacenters of exchanges and other trading venues, which typically contain both the exchange’s computer system and the systems of HFT firms and other algorithmic traders, of banks, of communications suppliers, and so on.¹⁰ Exchanges’ datacenters aren’t generally found in city centers, but in suburban areas in which real estate is cheaper. The datacenters important to HFT are mostly large buildings, and indeed they usually look like suburban warehouses, with, for example, few windows. They are packed with tens of thousands of computer servers, typically on racks in wire-mesh cages (although sometimes the cages have opaque walls, so that a trading firm’s competitors cannot see the equipment it is using). The servers are interconnected by mile upon mile of cabling, typically running above the racks in what looks to an outsider like an incomprehensibly complex spaghetti of different types of cable. In aggregate, those servers consume very large quantities of electricity and generate large amounts of heat, making a powerful cooling system also a requisite. Normally, few human beings are to be found in these datacenters, just a small number of security and maintenance staff, along with (at least some of the time) engineers from exchanges, trading firms, or communications suppliers who may be visiting to fix problems or install new equipment.

FIGURE 1.3. The “equities triangle” in New Jersey. The Nasdaq and NYSE (New York Stock Exchange) datacenters host the share-trading exchanges run by those groups; NY4 and NY5, which are in effect a single datacenter, host the s...