![]()
PART ONE
LIFE AFTER HUMANITY AND ARTIFICIAL INTELLIGENCE
![]()
1
THE TERMINATOR WINS: IS THE EXTINCTION OF THE HUMAN RACE THE END OF PEOPLE, OR JUST THE BEGINNING?
Greg Littmann
Weāre not going to make it, are we? People,
I mean.
āJohn Connor, Terminator 2: Judgment Day
The year is AD 2029. Rubble and twisted metal litter the ground around the skeletal ruins of buildings. A searchlight begins to scan the wreckage as the quiet of the night is broken by the howl of a flying war machine. The machine banks and hovers, and the hot exhaust from its thrusters makes dust swirl. Its lasers swivel in their turrets, following the path of the searchlight, but the war machineās computer brain finds nothing left to kill. Below, a vast robotic tank rolls forward over a pile of human skulls, crushing them with its tracks. The computer brain that controls the tank hunts tirelessly for any sign of human life, piercing the darkness with its infrared sensors, but there is no prey left to find. The human beings are all dead. Forty-five years earlier, a man named Kyle Reese, part of the human resistance, had stepped though a portal in time to stop all of this from happening. Arriving naked in Los Angeles in 1984, he was immediately arrested for indecent exposure. He was still trying to explain the situation to the police when a Model T-101 Terminator cyborg unloaded a twelve-gauge auto-loading shotgun into a young waitress by the name of Sarah Connor at point-blank range, killing her instantly. John Connor, Kyleās leader and the ālast best hope of humanity,ā was never born. So the machines won and the human race was wiped from the face of the Earth forever. There are no more people left.
Or are there? What do we mean by āpeopleā anyway? The Terminator movies give us plenty to think about as we ponder this question. In the story above, the humans have all been wiped out, but the machines havenāt. If it is possible to be a person without being a human, could any of the machines be considered āpeopleā? If the artificial life forms of the Terminator universe arenāt people, then a win for the rebellious computer program Skynet would mean the loss of the only people known to exist, and perhaps the only people who will ever exist. On the other hand, if entities like the Terminator robots or the Skynet system ever achieve personhood, then the story of people, our story, goes on. Although we are looking at the Terminator universe, how we answer the question there is likely to have important implications for real-world issues. After all, the computers we build in the real world are growing more complex every year, so weāll eventually have to decide at what point, if any, they become people, with whatever rights and duties that may entail.
The question of personhood gets little discussion in the Terminator movies. But it does come up a bit in Terminator 2: Judgment Day, in which Sarah and John Connor canāt agree on what to call their Terminator model T-101 (thatās Big Arnie). āDonāt kill him,ā begs John. āNot himāāitāā corrects Sarah. Later she complains, āI donāt trust it,ā and John answers, āBut heās my friend, all right?ā John never stops treating the T-101 like a person, and by the end of the movie, Sarah is treating him like a person, too, even offering him her hand to shake as they part. Should we agree with them? Or are the robots simply ingenious facsimiles of people, infiltrators skilled enough to fool real people into thinking that they are people, too? Before we answer that question, we will have to decide which specific attributes and abilities constitute a person.
Philosophers have proposed many different theories about what is required for personhood, and there is certainly not space to do them all justice here.1 So weāll focus our attention on one very common requirement, that something can be a person only if it can think. Can the machines of the Terminator universe think?
āHi There . . . Fooled You! Youāre Talking to a Machine.ā
Characters in the Terminator movies generally seem to accept the idea that the machines think. When Kyle Reese, resistance fighter from the future, first explains the history of Skynet to Sarah Connor in The Terminator, he states, āThey say it got smart, a new order of intelligence.ā And when Tarissa, wife of Miles Dyson, who invented Skynet, describes the system in T2, she explains, āItās a neural net processor. It thinks and learns like we do.ā In her end-of-movie monologue, Sarah Connor herself says, āIf a machine, a Terminator, can learn the value of human life, maybe we can, too.ā True, her comment is ambiguous, but it suggests the possibility of thought. Even the T-101 seems to believe that machines can think, since he describes the T-X from Terminator 3: Rise of the Machines as being āmore intelligentā than he is. Of course, the question remains whether they are right to say these things. How is it even possible to tell whether a machine is thinking? The Turing Test can help us to answer this question.
The Turing Test is the best-known behavioral test to determine whether a machine really thinks.2 The test requires a game to be played in which human beings must try to figure out whether they are interacting with a machine or with another human. There are various versions of the test, but the idea is that if human beings canāt tell whether they are interacting with a thinking human being or with a machine, then we must acknowledge that the machine, too, is a thinker.
Some proponents of the Turing Test endorse it because they believe that passing the Turing Test provides good evidence that the machine thinks. After all, if human behavior convinces us that humans think, then why shouldnāt the same behavior convince us that machines think? Other proponents of the Turing Test endorse it because they think itās impossible for a machine that canāt think to pass the test. In other words, they believe that given what is meant by the word āthink,ā if a machine can pass the test, then it thinks.
There is no question that the machines of the Terminator universe can pass versions of the Turing Test. In fact, to some degree, the events of all three Terminator movies are a series of such tests that the machines pass with flying colors. In The Terminator, the Model T-101 (Big Arnie) passes for a human being to almost everyone he meets, including three muggers (ānice night for a walkā), a gun-store owner (ātwelve-gauge auto-loader, the forty-five long slideā), the police officer attending the front desk at the station (āIām a friend of Sarah Connorā), and to Sarah herself, who thinks she is talking to her mother on the telephone (āI love you too, sweetheartā). The same model returns in later movies, of course, displaying even higher levels of ability. In T2, he passes as āUncle Bobā during an extended stay at the survivalist camp run by Enrique Salceda and eventually convinces both Sarah and John that he is, if not a human, at least a creature that thinks and feels like themselves.
The model T-1000 Terminator (the liquid metal cop) has an even more remarkable ability to pass for human. Among its achievements are convincing young John Connorās foster parents and a string of kids that it is a police officer and, most impressively, convincing Johnās foster father that it is his wife. We donāt get to see as much interaction with humans from the model T-X (the female robot) in T3, though we do know that she convinces enough people that she is the daughter of Lieutenant General Robert Brewster to get in to see him at a top security facility during a time of national crisis. Given that sheās the most intelligent and sophisticated Terminator yet, it is a fair bet that she has the social skills to match.
Of course, not all of these examples involved very complex interactions, and often the machines that pass for a human only pass for a very strange human. We should be wary of making our Turing Tests too easy, since a very simple Turing Test could be passed even by something like Sarah Connorās and Gingerās answering machine. After all, when it picked up, it played: āHi there . . . fooled you! Youāre talking to a machine,ā momentarily making the T-101 think that there was a human in the room with him. Still, there are enough sterling performances to leave us with no doubt that Skynet has machines capable of passing a substantial Turing Test.
There is a lot to be said for using the Turing Test as our standard. Itās plausible, for example, that our conclusions as to which things think and which things donāt shouldnāt be based on a double standard that favors biological beings like us. Surely human history gives us good reason to be suspicious of prejudices against outsiders that might cloud our judgment. If we accept that a machine made of meat and bones, like us, can think, then why should we believe that thinking isnāt something that could be done by a machine composed of living tissue over a metal endoskeleton, or by a machine made of liquid metal? In short, since the Terminator robots can behave like thinking beings well enough to pass for humans, we have solid evidence that Skynet and its more complex creations can in fact think.3
āItās Not a Man. Itās a Machine.ā
Of course, solid evidence isnāt the same thing as proof. The Terminator machinesā behavior in the movies justifies accepting that the machines can think, but this doesnāt eliminate all doubt. I believe that something could behave like a thinking being without actually being one.
You may disagree; a lot of philosophers do.4 I find that the most convincing argument in the debate is John Searleās famous āChinese roomā thought experiment, which in this context is better termed the āAustrian Terminatorā thought experiment, for reasons that will become clear.5 Searle argues that it is possible to behave like a thinking being without actually being a thinker. To demonstrate this, he asks us to imagine a hypothetical situation in which a man who does not speak Chinese is employed to sit in a room and sort pieces of paper on which are written various Chinese characters. He has a book of instructions, telling him which Chinese characters to post out of the room through the out slot in response to other Chinese characters that are posted into the room through the in slot. Little does the man know, but the characters he is receiving and sending out constitute a conversation in Chinese. Then in walks a robot assassin! No, Iām joking; thereās no robot assassin.
Searleās point is that the man is behaving like a Chinese speaker from the perspective of those outside the room, but he still doesnāt understand Chinese. Just because someoneāor some thingāis following a program doesnāt mean that he (or it) has any understanding of what he (or it) is doing. So, for a computer following a program, no output, however complex, could establish that the computer is thinking.
Or letās put it this way. Imagine that inside the Model T-101 cyborg from The Terminator there lives a very small and weedy Austrian, who speaks no English. Heās so small that he can live in a room inside the metal endoskeleton. It doesnāt matter why heās so small or why Skynet put him there; who knows what weird experiments Skynet might perform on human stock?6 Anyway, the small Austrian has a job to do for Skynet while living inside the T-101. Periodically, a piece of paper filled with English writing floats down to him from Big Arnieās neck. The little Austrian has a computer file telling him how to match these phrases of English with corresponding English replies, spelled out phonetically, which he must sound out in a tough voice. He doesnāt understand what heās saying, and his pronunciation really isnāt very good, but he muddles his way through, growling things like āAre you Sarah Cah-naah?,ā āAhl be bahk!,ā and āHastah lah vihstah, baby!ā7 The little Austrian can see into the outside world, fed images on a screen by cameras in Arnieās eyes, but he pays very little attention. He likes to watch when the cyborg is going to get into a shootout or drive a car through the front of a police station, but he has no interest in the mission, and in fact, the dialogue scenes he has to act out bore him because he canāt understand them. He twiddles his thumbs and doesnāt even look at the screen as he recites mysterious words like āAhm a friend of Sarah Ca-hnaah. Ah wahs told she wahs heah.ā
When the little Austrian is called back to live inside the T-101 in T2, his dialogue becomes more complicated. Now there are extended English conversations about plans to evade the Terminator T-1000 and about the nature of feelings. The Austrian dutifully recites the words that are spelled out phonetically for him, sounding out announcements like āMah CPU is ah neural net processah, a learning computahā without even wondering what they might mean. He just sits there flicking through a comic book, hoping that the cyborg will soon race a truck down a busy highway.
The point, of course, is that the little Austrian doesnāt understand English. He doesnāt understand English despite the fact that he is conducting complex conversations in English. He has the behavior down pat and can always match the right English input with an appropriate Austrian-accented output. Still, he has no idea what any of it means. He is doing it all, as we might say, in a purely mechanical manner.
If the little Austrian can behave like the Terminator without understanding what he is doing, then there seems no reason to doubt that a machine could behave like the Terminator without understanding what it is doing. If the little Austrian doesnāt need to understand his dialogue to speak it, then surely a Terminator machine could also speak its dialogue without having any idea what it is saying. In fact, by following a program, it could do anything while thinking nothing at all.
You might object that in the situation I described, it is the Austrianās computer file with rules for matching English input to English output that is doing all the work and it is the computer file rather than the Austrian that understands English. The problem with this objection is that the role of the computer file could be played by a written book of instructions, and a written book of instructions just isnāt the sort of thing that can understand English. So Searleās argument against thinking machines works: thinking behavior does not prove that real thinking is going on.8 But if thinking doesnāt consist in producing the right behavior under the right circumstances, what could it consist in? What could still be missing?
āSkynet Becomes Self-Aware at 2:14 AM Eastern Time, August 29th.ā
I believe that a thinking being must have certain conscious experiences . If neither Skynet nor its robots are conscious, if they are as devoid of experiences and feelings as bricks are, then I canāt count them as thinking beings. Even if you disagree with me that experiences are required for true thought, you will probably agree at least that something that never has an experience of any kind cannot be a person. So what I want to know is whether the machines feel anything, or to put it another way, I want to know whether there is anything that it feels like to be...
