Planet Google

One Company's Audacious Plan to Organize Everything We Know

Randall Stross

• Publisher: Free Press
• Year: 2008
• Print ISBN: 9781416546962
• eBook ISBN: 9781416580478
• Topic: Business
• Subtopic: Business General
• Index: Business

Based on unprecedented access he received to the highly secretive "Googleplex, " acclaimed New York Times columnist Randall Stross takes readers deep inside Google, the most important, most innovative, and most ambitious company of the Internet Age. His revelations demystify the strategy behind the company's recent flurry of bold moves, all driven by the pursuit of a business plan unlike any other: to become the indispensable gatekeeper of all the world's information, the one-stop destination for all our information needs. Will Google succeed? And what are the implications of a single company commanding so much information and knowing so much about us? As ambitious as Google's goal is, with 68 percent of all Web searches (and growing), profits that are the envy of the business world, and a surplus of talent, the company is, Stross shows, well along the way to fulfilling its ambition, becoming as dominant a force on the Web as Microsoft became on the PC. Google isn't just a superior search service anymore. In recent years it has launched a dizzying array of new services and advanced into whole new businesses, from the introductions of its controversial Book Search and the irresistible Google Earth, to bidding for a slice of the wireless-phone spectrum and nonchalantly purchasing YouTube for $1.65 billion. Google has also taken direct aim at Microsoft's core business, offering free e-mail and software from word processing to spreadsheets and calendars, pushing a transformative -- and highly disruptive -- concept known as "cloud computing." According to this plan, users will increasingly store all of their data on Google's massive servers -- a network of a million computers that amounts to the world's largest supercomputer, with unlimited capacity to house all the information Google seeks. The more offerings Google adds, and the more ubiquitous a presence it becomes, the more dependent its users become on its services and the more information they contribute to its uniquely comprehensive collection of data. Will Google stay true to its famous "Don't Be Evil" mantra, using its power in its customers' best interests? Stross's access to those who have spearheaded so many of Google's new initiatives, his penetrating research into the company's strategy, and his gift for lively storytelling produce an entertaining, deeply informed, and provocative examination of the company's audacious vision for the future and the consequences not only for the business world, but for our culture at large.

CHAPTER 1

Open and Closed

War, hyperinflation, breakdowns in public utility services. None of these elements of an adverse business environment have fallen into Google’s path. The company has had the good fortune to enjoy a most hospitable business environment. It is not widely appreciated, however, how dependent Google is upon an environment remaining free of not only major disruptions, particular to the online world, but also other problems that would bring its business to a standstill as surely as a war would. Google needs the Internet to remain open and true to its founding spirit, without (pay) walls, without (subscription) fences, without (proprietary software) barriers, without any other impediments to the unrestricted exchange of information.

The credo that holds that “information wants to be free” has always faced an opposing school of thought: “Information is too valuable to be free.” From this latter perspective, information is a commercially valuable asset, to be hoarded, not shared. Access to information is unimpeded in the open camp, and severely restricted in the closed camp. The epitome of open is “wiki” sites, which are completely open to anyone to edit, as well as read. Their opposites are social networking sites like Facebook, which permit only members to enter, limiting access to information to subsets of a member’s web of friends.

Google’s search engine needs access to the entire Internet, not merely the patches that remain outside the walled gardens of social networking sites. The company’s very existence depends upon the advocates of an open online environment holding at bay the threat of encroachment by their opponents.

Contention between open and closed is also the defining issue roiling the world of software development. Microsoft achieved its success as a practitioner of the closed approach to software development, keeping its source code secret and using its control of the operating system to extend its reach into other sectors of the software business. It treated industry standards that belonged to no single company as competitors to Microsoft’s own proprietary software. The company follows a strategy of what it calls “embrace and extend,” that is, begin with industry standards but “extend” them by attaching Microsoft’s own proprietary additions to them.

Microsoft’s self-aggrandizement antagonized many companies and software developers in the industry, and ultimately it created a backlash, the open-source software movement, which has become a potent challenge to Microsoft’s closed approach to software. Open-source software permits software developers to see all of a program’s source code, something that Microsoft would never allow. The movement depends upon software developers’ willingness to volunteer their time and skills, without remuneration, but hatred of Microsoft and all it stands for has provided ample motivation.

Google uses open-source software extensively for its own operations. It has not, however, placed its own proprietary search formulas into the public domain. It’s a company that is attached to its own secrets and will not win any contests for corporate transparency. Still, compared to Microsoft, Google is more closely aligned to those in the industry clustered around the open model. The two companies represent not just the interests of their own firms but also the interests of two ideologically opposed camps, open and closed. It was fitting that it was Microsoft that won the privilege in late 2007 of investing in Facebook: the two companies manifest the greatest comfort with closed over open. Their alignment unites them in fighting their principal ideological rival, Google.

The social networking phenomenon attracts inordinate attention now because of its recent popularity. The World Wide Web, Google’s original domain, has been a presence for so long that it has receded into the background. But as long as open and closed models contend for favorable position, the Web’s founding principles remain as relevant to the present moment as ever. The Web was conceived as an alternative to closed communications systems. It was built as an open medium—open to anyone to publish or read, designed not only to make information easy to access, but also to make tracing the origins of an idea easy, as simple as clicking on a link. No geographic boundaries, no fences, nothing to impede the researcher from zipping across the Internet, link by link, to find information that was both useful and free, wherever it happened to be, anywhere in the world.

It was conceived by an academic for fellow academics, and the only place the Web existed at first was on servers that belonged to research laboratories. Tim Berners-Lee, a computer scientist who was on the staff of CERN, the European Organization for Nuclear Research, in Geneva, Switzerland, came up with the essential concepts in 1989 and built the first servers for holding Web pages in 1990. From CERN, the Web spread to other particle physics laboratories. The Web browsers at that time worked only on scientific workstations, and the Web could easily have remained what it was then, a tool used within the closed world of physicists. In 1993, however, CERN announced that its Web technologies could be used by anyone, without paying royalties. It set in motion the creation of the world’s largest open network—and ultimately the creation of Google, the most valuable company to be built upon that open network.

The combination of open and network was an oxymoron in the 1990s, when commercial-grade networks were closed, by design. Cellular phone networks and cable television networks, for example, ensured quality by using a strong central authority to exert close control and restrict who and what had access to the network. In contrast, the Web lacked a central authority and also lacked built-in mechanisms for Web site publishers to collect revenue, so initially the Web appeared to be the place for academics, amateur authors, and others without interest in commercial pursuits to share information without concern about remuneration.

In its openness and radically decentralized design, the Web mirrored the design of the underlying technical standards for the Internet, the networking technologies that are used to transfer digital bits of information. But the Web was not an instant success. In June 1993, two years after it was introduced, less than one-half of 1 percent of all Internet traffic was used for Web pages—the rest of the traffic was for e-mail, file transfers, and discussions in “news groups.” There were not many places on the Web to go: only 130 or so sites were in existence then. In 1994, a year after the introduction of Mosaic, the first popular Web browser, there were twenty-seven hundred sites, but Web traffic still was only 6 percent of all Internet traffic. The Web remained a curiosity.

The Web would have remained a place for academics with very specialized information needs were it not for the willingness of more and more individuals outside of academe to place information on the Web for free. Today’s Google employees and shareholders should be eternally grateful to the Web’s early contributors, because later demand for Google’s search service came about only because so many people decided, one by one, to give information away without charge on the Web.

Before the Web, information was available online, but one had to pay dearly for access. The first generation of digital information services were commercial information providers, such as Lockheed’s Dialog service, which began in the 1970s and charged high prices to business clients. The second generation of pre-Web information services were sold to consumers by American Online (AOL), CompuServe, and Prodigy. They offered customers access to their private networks, reached with a dial-up modem using the household’s phone line, and information that was available only to their members.

No one in the early 1990s could have predicted that the Web would attract abundant information that was both of high quality and free. The only business model for information services that seemed practical was to build a walled garden around information and then charge users to enter the garden. Google enjoyed a huge advantage arriving when it did in the late 1990s, as a latecomer to the digital information business. It was conceived outside the walled garden and never had to pass through a painful transition, as did its forebears, replacing the core business model based on a closed network for one based on an open one.

Microsoft had the ill luck to become interested in information services later than incumbents like AOL but not late enough to grasp the competitive threat to closed networks that the Web would soon pose. Six years before Google’s founding, Microsoft began planning, in 1992, to offer its own information service, which would eventually appear as MSN (Microsoft Network) in 1995. It naturally followed the closed-network model of AOL and the others. Nathan Myhrvold, Microsoft’s research chief at the time, likened MSN’s future role to that of a shopping mall landlord. Other companies would be invited to sell information or goods within the Microsoft Network, and Microsoft would retain 10 to 30 percent of the revenue. When the growth of the Internet loomed as a free alternative to the walled garden, he held out hope that Microsoft could still set up a toll booth outside MSN’s garden by inserting Microsoft’s proprietary software into all e-commerce purchases on the Internet, and exact a transaction fee of 1 or 2 percent, just as Visa or MasterCard did.

The one company that seemed to be ideally positioned to profit from the advent of the Web was Netscape Communications, whose Netscape Navigator Web browser became sensationally popular upon release in 1994. The company grew so fast that in August 1995, only eighteen months after its founding, it had one of the most successful initial public offerings for any company in the technology industry. Netscape declared itself to be the faithful guardian of “openness,” the foundation of the Internet’s architecture, and the company placed the internal source code for its browser into the public domain in 1998. It learned that giving away its Web browser for free was easy. Collecting revenue from paying customers, however, was far more difficult: it required offering a product that embodied intellectual property that was not freely available, and when Netscape tried to sell Web server software to corporate customers, it found that sales came slowly. It attempted, in vain, to have its proprietary additions to Web standards adopted as a new industry standard. As a chronicle of Netscape’s history summed up, Netscape was “open, but not open.”

Almost every information technology company claims to be a champion of open standards—even Microsoft did so, too, touting Windows as an open platform, that is, open to any software developer who wished to create Windows applications, without having to secure Microsoft’s permission and without having to share the revenue with Microsoft (in contrast to game system manufacturers likes Nintendo, which control their own closed software systems and force game developers to share revenue from game sales). When Microsoft added its own proprietary code to the industry’s Internet standards, such as internal Web page tags that its Internet Explorer would recognize, but Netscape’s browser would not, it euphemistically called the changes “extensions” to preserve the appearance of adhering to open standards.

When Google was founded in 1998, it did not have to fight against Microsoft’s extensions, nor did it have to displace Microsoft’s operating system or push back Microsoft’s browser. It had the advantage of a better field position than previous Microsoft challengers had ever enjoyed—it floated above the fray. Any browser, running on any operating system, could reach Google.

Google was fortunate in another respect: the sheer mass of pages placed on the Web overwhelmed the abilities of Yahoo and others in the first generation of Web navigation and search businesses to develop search techniques that grew in sophistication as fast as the Web was growing. Their failure created the opportunity for Google.

Craig Silverstein, the first Google employee hired in 1998 by the two founders, later said that had the company been founded two years earlier, or even one year earlier, it surely would have failed. Before 1998, the Web was still small enough that almost any search method served well enough, producing a list of sites with matching Web pages that was short enough to be easy to scan in its entirety. By 1998, however, the Web was much larger and a need had arisen for a search engine that could do more than simply match the text of the search term with that of all Web pages that contained the phrase. Silverstein said it also had to “discriminate between good results and not-so-good results.”

Google would have no search service to offer if Web pages were inaccessible to its “spider,” the software that systematically “crawled” the Web, collecting copies of Web pages that were then indexed and analyzed, in readiness for matching when a visitor to Google later submitted a search request. (As fast as Google’s software appears to perform a search, it should be remembered that when Google receives a search request, its search does not at that moment check the world’s Web sites, but rather checks the copies of those sites that were collected earlier and stored on Google’s servers.) When the crawling software was written in unsophisticated form, as Google’s initial version was, it caused many problems for the Web sites it crawled. In some cases, where bandwidth was limited, the Google crawlers’ visits resulted in a spike in traffic that the Web site could not accommodate. The software that ran the Web site would freeze, closing the site to all visitors. This did not endear Google to the sites’ owners, some of whom sent Brin and Page angry e-mail messages or phoned them to convey their objections.

The upset passed. The code embedded in the Google spider was improved, which reduced the time that it spent on each site and also reduced the likelihood that it would crash the system it was visiting. At the same time, Web hosts became accustomed to the visits of automated software programs and understood spiders’ visits would, in turn, make their sites visible to the search engines and could serve in the future to draw in visits by actual humans.

The Web’s original designers agreed upon a piece of code Web site hosts could use to signal that a spider or any other kind of “robot” software was not welcome to visit. It was Google’s very good fortune that the Web grew without Web site owners’ electing to use this option to block visits by Google’s spider. The relevance of Google’s search results are dependent upon having access to the broadest possible range of Web pages. These provide the essential materials used in calculations that go into Google’s ranking of search results, placing Web pages with the greatest presumed authority at the top of the list.

To calculate authority for any given page, Google’s software looks at its database of links for the entire Web and notes which Web sites link to that page, providing, in effect, a recommendation. This provides only a beginning. The sites that provide recommendations have to be examined carefully, in order to determine whether the recommendation should be weighted heavily or lightly or disregarded entirely. The software returns to the database of links to see who recommends the recommending sites, and when those are found, to see who recommends those sites, and so on. The process works backwards, recursively. Checking who points to whom may seem as endless a process as going up a stairway in an Escher print, but the long chain of calculations eventually produces a distillation of relative authority for every page on the Web, which is expressed as a number, 1 to 10. It sums up whether editors at other Web sites regard any one Web page as authoritative and worthy of recommending to others. Google refers to the number as PageRank (the Page officially refers to Larry Page, who developed the original formula, but his surname permits the term to work nicely even if its paternity remains unknown to the reader). PageRank wasn’t the first system devised to analyze the Web’s structure—Cornell computer scientist Jon Kleinberg’s mid-1990s work at the IBM Almaden Research Center is generally credited as a landmark in the field—but PageRank was the one that made it out of the lab first.

With its near-endless tracing of links and cross-references, PageRank relies upon a complete database of all links found on pages in the entire Web, and that in turn requires that Google’s crawler be able to range freely across an open ecosystem. Google can utilize the judgments embodied in links without having to purchase rights to use them because of the open ethos of the Web. If even a small number of owners of Web sites with high PageRank scores and presumed high authority were to exclude Google’s spider—demanding, perhaps, that Google share revenue earned by indexing their sites—then Google’s ability to operate as it has would end.

The advent in the 1990s of an open model for publishing information on the Web did not immediately spell doom for the closed model of walled gardens. Novice users of e-mail and the Web were slow to venture out beyond the carefully manicured garden of their Internet service provider. This gave the service provider considerable leverage in negotiating deals with prospective tenants who wanted to set up shop within the proprietary network. The most sought after real estate online was screen space within the gated world of AOL, a fact that AOL exploited to the fullest in negotiations with prospective commercial tenants.

AOL was the widely acknowledged gatekeeper to online information in the place that its members found most convenient to access. The fact that the area within its garden was finite and merely a subset of what was available on the Web did not seem to bother its members. An observation that Vic Gundotra, a former Microsoft manager now at Google, made about Windows applied to AOL too: “At Microsoft, our view was that if the walled garden was big enough, it was indistinguishable from something that was open.”

When AOL customers sought faster Internet service than AOL’s dial-up service and switched to cable or DSL broadband service, they also began to see the limitations of AOL’s information services. AOL’s members began declining after 2001, falling from about 28 million membership in the United States to 9.3 million at the end of 2007. As it lost members, AOL executives realized that its wall prevented it from relying more heavily on advertising revenue to replace shrinking subscription revenue. In 2005, Michael Kelly, an ad-sales executive placed in charge of all of AOL’s Web properties, complained, “My biggest problem is the walled garden. The world can’t see the good stuff we do every day.” AOL edged closer to the outside world when it announced a strategic alliance with Google in late 2005. The next year, AOL finally tore down its own garden wall.

The opening of AOL’s closed network seemed to mark the permanent end of an era. Small vestigial gardens offering content from a single source, open to subscribers only, could be found here and there, but one by one, they, too, acknowledged the greater attractiveness of an advertising-based business model open to all visitors. The New York Times’s site experimented for a year with a “pay wall” that was placed around some of its columnists, and decided to remove the wall in 2007. The Wall Street Journal’s Web site remains the last major outpost of a gated, subscribers-only community.

At the same time, however, that the triumph of the open network model seemed complete, Facebook demonstrated that a closed network model could still work well—very well, indeed. In 2007, it grew into the second-largest social network site in the United States, with more than 42 million members by October—only MySpace, the sensation of 2006, w...

Table of contents

• Cover
• ALSO BY RANDALL STROSS
• Copyright
• Dedication
• Contents
• Introduction
• 1. Open and Closed
• 2. Unlimited Capacity
• 3. The Algorithm
• 4. Moon Shot
• 5. GooTube
• 6. Small World, After All
• 7. A Personal Matter
• 8. Algorithm, Meet Humanity
• Conclusion
• Notes
• Acknowledgments
• About the Author