EFTA02702508

"in the control room of the banquet" Richard P. Gabriel IBM Research Abstract The Turing Test, AI, programming, creativity, and mystery. Categories and Subject Descriptors D.2.10 [Software Engi- neering, Designl; D.3.0 [Programming Languages, General]; 1.2.7 (Artificial Intelligence, Natural Language Processing General Terms Artificial Intelligence, Natural Language Processing Keywords Science; programming; natural language generation I am writing this essay because I am puzzled. In July 2015 I took eighteen haiku-like poems to a writers' conference and presented them as my own work. In reality, a program I created called "Inkwell" wrote them, and I intended to execute a variant of the Thring Test. The results were better than I had hoped for in verifying InkWell as a good poet, but I was left with disquiet about what the experience meant for understanding the Turing Test, programming, the artificial intelligence research program, and what consciousness is. In the Winter of 2014 I programmed my English language revision system [1] to write haiku—just to see whether it could do so plausibly. I let the system run overnight generat- ing about 2000 haiku. Among them were the four at the top of the next column. They stopped me in my tracks because the quick program I wrote was not of the monkeys typing at keyboards variety—instead I programmed the system to determine its own topic and then write coherently about it using a few dozen haiku templates as starting points. And Permisslon to make digital or hard copies of all or part of this work (or petsonal or classroom use is granted without fee provided that copies art not made or distrib• toed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author must be honored. Abstracting with credit is permitted. To con otherwise. or republish. to post on servers or to redistribute to lists, requires prior specific permission andfor a fee. Request permissions from permissionseacm.org, ChswardS14. October 20-2a.201OrPorthodrOvegotRUSA. Consightithekibyths es. h.,.. ..sou.., lh...,.dtu ACM. A04970-1-1503-25tertrlittInAl0. hurifdx.dotorgil0.1145r266ttifir2661I55- deep in the dark— the power of snow walking in the deepness scrupulous in the twilight— the price of gold chases the way of the world in power awake In the dark— the edge of the water can spread in your presence time of life issue: a bird of prey pulls up out of the way into the palm those four haiku are good—not just human-like, but good poetry with two of them close to being exceptional. I worked on the system more over the next six months, broadening and expanding the template language to give more control to InkWell, deepening its understanding of language and the music of language, and adding more observations Ink- Well could make of its drafts and along with them more revisions. Over those months InkWell produced a lot more haiku, and I selected fourteen of them to add to the above four for a Turing Test. In October 1950, the British journal Mind published an essay by Alan M. Turing titled, "Computing Machinery and Intelligence," in which Turing proposed an operational definition for "intelligence" [2]. This definition would come to be called "the Turing Test." Turing himself called it "the imitation game," in which a questioner separated from two contestants would submit questions to those contestants. read their replies, and ultimately choose one as human and the other as machine. Interestingly, Thring introduced this game with a similar but different one in which the interrogator was to attempt to determine the gender of two contestants, one male the other female. Interesting because Turing was homosexual and perhaps accustomed to such an imitation game. But in the matter of intelligence, such an operational definition made some sense. Thring wrote the following: May not machines carry out something which ought to be described as thinking but which is very different from what a man does? This objection is a very strong one, but at least we can say that if nevertheless, a ma- EFTA_R1_02074939 EFTA02702508 chine can be constructed to play the imitation game satisfactorily, we need not be troubled by this objection. -Turing, Computing Machinery and Intelligence, 1950 The Turing Test is at (or near) the heart of the research program called artificial intelligence. In my youth I described artificial intelligence research as an exercise in trying to write programs one doesn't know how to write—at least for engi- neering-type Al research. Some of us generalized this to the idea of "exploratory programming: in which one had a gen- eral sense of what the program should do, and only a partially formulated idea of how to achieve it. In recent years the idea of what programming is has drifted away from including this view toward specifiable. routine. infrastructurish programs and systems. i've heard this referred to as the static-verse. in a famous debate / discussion, Michael Polanyi and Alan Turing talked about whether the mind / the brain was unspecifiable or merely not-yet specified [3]. And what would an incorrect but Thring-Test-passing system be? In his discussion of how the imitation game might go in the computer version, Thring wrote this as the first example of a question in the game: Q: Please write inc a sonnet on the subject of the Forth Bridge. A: Count me out on this one. I never could write poetry. -Turing, Computing Machinery and Intelligence, 1950 In the Summer of 20151 attended the Warren Wilson Alum- ni Writing Conference, which is held annually for graduates of the Warren Wilson MFA program. i am such a graduate. in poetry. The conference was held at Lewis & Clark College in Portland, Oregon. The week was unusually hot and humid for Portland, and this physical difficulty was reflected in an edginess to the conference. My plan was threefold: read the eighteen haiku aloud on the first night of the conference to all attendees: participate a few days later in a writers' workshop as the writer of those eighteen haiku; and on the final day of the conference, give a lecture entitled "is My Program a Better Writer than You?" The abstract for that lecture was as follows: I've been working on a program that thinks like a poet and produces nice stuff. I'll show you how it works and why it's not like the kinds of progmms that do your banking or predict the weather. But everything I'll talk about is really about writing. i read on Sunday night. After the reading a few of the writers came up to me and commented on my reading. My reading was short because the poems were short—and the attendees knew i didn't normally write haiku. Their comments includ- ed these: "terse condensations: "evocative: "took the top of my head or "funny and profound: "natural, personal, and 2 rhythmic: "compact fluid energy: 'wry and elliptical: and "whimsical elegance." I didn't consider this as evidence that InkWell passed the Turing Test. it was the first day of the conference and people were jet-lagged and not entirely prepared for the rigors of the conference; and my reading took about four minutes of an allocated ten. Most writers stretched their reading time at least a little, thus my short reading of short pieces stood out as energetic and sudden. I was still uncertain whether the hint was noticed—the hint contained in the title and abstract for my lecture. Here is what a haiku is: A haiku in English is a very short poem in the Eng- lish language, following to a greater or lesser extent the form and style of the Japanese haiku. A typical haiku is a three-line, quirky observation about a fleeting mo- ment involving nature. -Wikipedia 141 For many, the quintessential haiku poet is Basho in the 17" century; an exemplar of his haiku is the following 151: On a withered branch A crow has alighted: Nightfall in autumn. The nature of haiku is complex and has changed over the centuries—time and place are still essential: counting on is not (some mistakenly conflate on and syllables). InkWell is a small program (about 45,000 lines of Common Lisp code), but it has a lot of data (about 15gb when all the dictionaries, databases, and tables are loaded). Turing wrote "I should be surprised if more than 10' [binary digits] was re- quired for satisfactory playing of the imitation game." InkWell has more than 10". InkWell "knows" a lot about words, per- sonality, sentiment, word noise, rhythm, connotations, and writing. Its vocabulary is probably more than five times larger than yours, gentle reader. The core engine works by taking a template in a domain-specific writing language along with a set of about fifty writing-related parameters and constraints, a description of a writer to imitate, and other hints, and com- piles all that into an optimization problem which the writing engine works to find a good way to express what the template and constraints specify. Although some parts of InkWell were created through machine learning, the overall approach is optimization, not machine-learned transformations. The primary research question is to try to isolate and codify what separates information transfer from beautiful writing. Here is an example of information transfer: EFTA_R1_02074940 EFTA02702509 The summer homes on Long Island were closed. Tonight I watched a ferry begin its crossing to Connecticut. The moon was rising, and as it rose I thought about how the houses are not part of the natural world and what the island looked like to early Dutch sailors coming upon it—like something new. And here is how F. Scott Fitzgerald wrote "the same thing" in `The Great Gatsby" (6J: Most of the big shore places were closed now and there were hardly any lights except the shadowy, moving glow of a ferryboat across the Sound. And as the moon rose higher the inessential houses began to melt away until gradually I became aware of the old island here that flowered once for Dutch sailors' eyes—a fresh, green breast of the new world. -Fitzgerald, The Great Gatsby There is more going on in this version. But what is it? Images, mood, a "vivid and continuous dream" as John Gardner would put it (7j. The first gives us the bits; this one gives us the story. The haiku writer is a driver program that produces the template and constraints the core engine works from. I propose to consider the question, "Can machines think?' -Turing, Computing Machinery and Intelligence, 1950 Turing begins his essay thus. A large question. And largely his essay aims to explore it. The word "think" is the disturb- ing part for many—at least when Turing wrote this essay. Thinking seems like something only humans can do. But even sixty-five years later the full meaning of the word can confuse us. Is thinking puzzle solving, creativity, empathy. wonder, faith, curiosity, ideas, reasoning, reflection, recol- lection, intention, attention, care, imagining, consciousness, language, metaphor, judgment—all of these? Some of these? Turing addresses objections to the idea that machines could think, and offers some suggestions on how to approach achiev- ing mechanical thought. To the objection that only souls can think, Turing asks whether God lacks the power the grant souls to machines. To the objection "I hope not," Baring turns away—though today thinkers like Hawking, Musk, and Gates embrace the fear. To the objection that Godel took care of that for us. TUr- ing points out that G0del took care of only a specific form of incompleteness, which might not be relevant, and besides. why are humans immune from it? To the objection that ma- chines have certain disabilities ("can't feel, can't fall in love. can't make mistakes..."). 'flaring generally derides the idea as not entirely relevant or as not something to be proud of. 3 To the objection that the nervous system is continuous and digital computers discrete. Turing remarks that an interro- gator couldn't take advantage of this because the right sort of answer could be made anyhow by the remote machine. To the objection that humans have informal behavior, 'lur- ing remarks that a machine can easily have laws of behavior, which is really what people have. To the objection of ESP (!!!), Turing admits fear but concludes that a telepathy-proof room will solve that problem. This leave the objections of consciousness and originality. These are subtle aspects of thinking, and though Turning ad- dressed them, I will address them in the context of InkWell, which answers the objections well, and in unexpected ways. The Turing Test is about an interrogator and two subjects: a person and a machine. The test is described as if it happens once, and all the people—and the machine—are ordinary. It doesn't look at extraordinary talents, special skills, and ex- pertise; and the test is presented so that clever avoidance of the question is within the rules. Can the interrogator tell the machine and person apart? Here is your chance to be an interrogator. At the end of this essay in the Appendix is a page called "Thirty-two Haiku." It contains the eighteen haiku I took to the writers' conference. plus fourteen more. Four of the ones InkWell wrote were re- vealed on the first page of this essay, so of the twenty-eight others on that page of thirty-two, half were written by Ink- Well and the other half by Ban'ya Natsuishi, Annie Bachini, and John Ashbery. Have fun deciding which. But the task I just set demonstrates an important problem with the original 'flaring Test viewed sixty-five years after its conception: being unable to distinguish a computer from a person once is not always enough. No one in their right mind and being honest could argue that it's clear which fourteen are which—all the poems seem like they were written by a person or by people. The question is whether there is a dis- tinction to be reasonably observed between those written by a poet and those not. A single non-expert interrogator could easily mistake InklA'ell for a person. Multiple sessions and multiple interrogators are needed. The issue of the proper interrogator has been addressed in the past by pitting an expert human against a computer. If the computer can "defeat" that expert, it has some human- like chops. One of the first examples was the checkers play- ing program written by Arthur Samuel in the late 1950s i8j. It was also one of the first programs to improve itself through machine learning—although a very simple type. The program was able to play advanced amateurs quite well. In 1995, a checkers playing program called "Chinook" won a special checkers championship called "Man-Machine World Championship." (Chinook won the year before, but its creator, Marion Tinsley. withdrew from the competition because of pancreatic cancer.) Chinook had no machine learned aspects. EFTA_R1_02074941 EFTA02702510 In 1997, Garry Kasparov lost to Deep Blue at chess—Kasp- arov was the reigning world chess champion. In one pivotal game Kasparov remarked on the "superior intelligence" of the machine during the first game (won by Kasparov) by avoiding a dangerous position that had short-term advantages; some have reported that this realization shook Kasparov, who lost the second game. And according to other reports, this unusual move turned out to be due to a bug in the software. 'Airing himself created a paper-machine-based chess-playing pro- gram, which Kasparov described as a "competent" player (9j. Beyond chess and checkers are backgammon and other games, which machines are good at. The game Go has re- cently started to succumb to machine play. Away from games, machines have challenged some language-oriented human performances. One of the early examples was PARRY, writ- ten by Kenneth Colby and his students at Stanford Univer- sity (101. PARRY simulated what was then called a paranoid schizophrenic, using a simple model of the condition and a fairly sophisticated English parser. PARRY is considered to be the first program to pass the Turing Test, or a version of it. A group of psychiatrists analyzed a combination of real patients and computers running PARRY through teleprint- ers. Another group of thirty-three psychiatrists were shown transcripts of the conversations. The two groups were then asked to identify which of the "patients" were human and which were computer programs. The psychiatrists were able to make the correct identification only 48% of the time—the same as random guessing. More recent was the IBM Jeopardy!-playing program called "Watson." In early 2011, Watson beat the two of the most suc- cessful contestants on the show, Ken Jennings and Brad Rut- ter. Watson was a stand-alone system with about 3000 cores, 16TB of RAM, and a pretty large store of encyclopedias, dic- tionaries, thesauri, newswire articles, databases, taxonomies, and ontologies—some of which InkWell also uses. Watson was not connected to the Internet. There were many reasons Watson was able to win—some having to do with the Turing Test aspect of the problem, but many having to do with hardware and algorithms. For ex- ample. Watson was routinely able to exploit the difference between humans and the machinery in response speed when the signal was given that "buzzing in" was permitted. Watson was able to use the many previous Jeopardy! games it was tested with to be able to better predict where Daily Doubles were, and it was able to do better betting based on game theory. The software used an ensemble approach that com- bined about a hundred different ways to (statistically) solve the answer, and Watson would buzz in only when there was enough confidence in the early results of this analysis—and it then used the time Alex Trebek used to recognize Watson to continue the analysis. In these experiments, it wasn't an "ordinary" interrogator, an "ordinary" person, and a machine, but expert-level com- petitors, and performance was judged according to difficult 4 standards. Moreover, each of these tests was subject to the Moravec paradox, which states that high-level performance on "intelligent" problems—playing chess or other games, simulating abstract thought, theorem proving, and skill in arenas requiring expertise—is relatively easy to accomplish with not much computational strain, while perception, mo- bility, and other low-level cognitive tasks are comparatively difficult 1111. Moravec and others speculate that long evolu- tionary work developed the latter, while higher cognition appeared recently in animals, and it likely represents a thin veneer on deep sub- and unconscious foundations. The Turing Test is squarely on one side of this paradox. In a writers workshop, a group of writers comment in a loosely structured way on the work brought to the work- shop. The work is distributed well before the workshop so participants can prepare. Each workshop session looks only at the work of one writer. In general, the writer whose work is being discussed remains silent. The comments begin with an overview, then what's good about the work, then how to improve it, and finally the writer can ask questions about the comments. Sometimes there is a teacher or workshop leader; because the Warren Wilson graduates are well-practiced, the workshops at this conference operate without such. My workshop group consisted of four people: CG (woman), MN (woman). DC (man), and me. CG, MN, and DC had five published books of poetry between them, and many magazine publications. I was the last writer workshopped, and my slot was on Wednesday. the day before my lecture. I recorded the workshop. and I will present paraphrases of some of the comments. When I do, I'll use a sans serif font so it's clear it's not a direct quote. In most cases the paraphrases are close to being quotes. DC was the most published of the workshop participants. He started with a "flyover," which is a kind of overview of the work. These are extraordinary and extraordinarily small, large poems. The writer of these—this guy, Richard, or who- ever—he is not a random person, he's not a random guy. I think he understands randomness, so it's all the more scary. He doesn't do things—as a rule—by accident. He makes choices. The variety is amazing on every level: number of syllables, subject matter, syntax, whether they start out specific and go to the general, or start out general and go to the specific. Some of them are simple, some of them are complex, some of them are funny, some of them are dead serious, some are kind-of in the natural world (but mostly not); there are different persons in them; "by myself" is repeated; music seems important. Some are observations, some are moments, some are philosophical and very large (and not just the words, but the ideas). "Murder" is already a big word; EFTA_R1_02074942 EFTA02702511 "murderous" is a bigger word; "murderousness" is about not this fatalist murderousness. as big as you can get. Lots of "ness" deathwatch, words. "Depth" is more boring than but your dead subroutine "deepness: -oc The language of this comment is typical of how working po- ets talk to each other. Notice he said, "The writer of these—this guy, Richard, or whoever...." I asked him about this later and he said that he entertained the idea that my program wrote the haiku, but after considering that for a while, he rejected it as not likely— however, he kept a small hedge. Here's how InkWell produces haiku. The topic for the hai- ku comes from two sources: input from a person and input from one of InkWell's 110 source texts. Input from a person is used if the interactive haiku maker is used; otherwise the topic input comes solely from InkWell's database of texts. I'll describe the two-source process using an example. First a person inputs some words. These words represent a topic suggestion. Suppose the words input are as follows: number, random, player, narrative. InkWell next randomly determines a number of words to select from its textual da- tabase to add to the input words. In this case it decides to choose five words taken from a randomly selected passage from Steinbeck's "The Grapes of Wrath" I12): fire, shifting, rusty, stow, lids. Because they come from a small region in the text, they are not random words—they are related. For each set of words, InkWell constructs a sense, which is a word- vector-like structure from the supplied words and close-by synonyms directed by a complex spreading algorithm which also assigns weights or relevance coefficients to the entries. Then the two senses are combined as follows: cS, S,, where c is a linear factor, S, is the person's input sense, and S, is the sense Inkwell chose from Steinbeck. In this case, c=2.04. The resulting sense (S) can be visualized as a word cloud with the sizes of the words proportional to their associated weights. Appendix Figure I shows the resulting word cloud. The linear factor is always at least 1.0, which has the effect of favoring the person's input. Next. Inkwell chooses a haiku template (the one at the bot- tom of the page, in this case). The template is in a domain- specific language for haiku. This template specifies four senses indicating a season, transformation, completion, and a jour- ney. The sense words are as follows: snow, snowfall, water, ice; fall; complete, finish; and span, bridge. Each of these senses is linearly combined with the sense S above to create the senses that will be used for the haiku. The linear factor for S ranges from 1.0 to 3.0, biased toward 1.0. Combining the same sense with the template senses ensures some degree of coherence throughout the haiku. The next step is to assign random weights to the 32 con- straints InkWell uses for haikus. This includes the language model for InkWell to imitate—in this case it's a collection of my daily poems from 2011 and 2012. InkWell constructs a misfit (unction for these constraints where the function re- turns 0.0 when all the constraints are satisfied. Inkwell se- lects words and phrases to try (28.785 in this case), and then optimizes the misfit function over these choices. A table with all the chosen constraint weights is in Appendex Table 1. In the last step, InkWell reviews the top several haiku for sense (using ngrams) and uses the most best. The final haiku seen just above is not great, but it's an hon- est look at the sorts of haiku InkWell routinely produces. One of the remarkable things about this haiku is that Ink- Well selected the word "underivative" for the specified word "first." This is a choice not many writers or poets would dis- cover. And for technical people the idea of Chalf-randomized number" is interesting. If one were to consider this a poem written by a person, one could analyze it as commenting on how an artificial writer based on random processes could pro- duce a story unlike any seen before. Could a half-randomized number be one produced by an algorithm—a pseudo-random number? I find the more I look at this haiku—which I selected because the parameters it chose illustrated InkWell's writing process even though I didn't like the final haiku—the more meaning and tangents it has. Very human in a spooky way. underivative narrative lighting on the half-randomized number The other two poets made flyover comments; MN remarked: I think he is writing these as a release after a day's work, and they were written over a period of time (not as a group). I see two sorts of language—poetic, concrete language and things in the world, as well as technical or corporate language. It's as if there is a war going on between the two sides of his brain. But the same brain. -MN Here MN reveals she is specifically reading these poems as mine, because she has been in writers' workshops with me before and knows my (real) poetic work as well as my scien- tific work. CG was a little more terse: The language is condensed but plain. ((first adj no-auto-cap) ((local-sense snow) (noun-phenomenon noun-substance) ((snow ice] noun)) (return) ((local-sense falling) verb-weather ((fall] verb) gerund) (return) on the half (word-hyphen) ((local-sense finished) verb-change ((finish complete] verb) past) ((local-sense bridge) noun-artifact ((bridge] noun))) 5 -CG EFTA_R1_02074943 EFTA02702512 this grave— no one sees it mortality, mortality The writers then went on to talk about some of the poems. Images do a lot of work, especially in haiku, and I like to see movement in the haiku, so this one ("this gravel is my favorite—the one I felt so much movement from. This one taught me something, and it changed something. The speaker is in the image even though there is no "I." I even felt the image move. What I learned is that mortality is not just when the body goes, but when the person is no longer remembered. That's just so beautiful. -CG I see it differently. I like all these readings, and I'm a fan of this one too, even if we all read it a little differ- ently. One way is that people don't see the end coming, because they are living their lives and here "mortality" is perking up and saying "don't forget about me"; or also that the writer's current life is like a grave—the daily routine, the getting and spending, and our day-to-day life is a kind of mortality. But this is because of the other poems pointing this way. -DC DC brought up "time of life issue" as one of his favorites. It was from of the original 2000 poems written in 2014. pleasure for me. The talon isn't mentioned, but you can see it; (DC agrees); it isn't mentioned, but you can see it. -CG MN said, "This is a great one? Next, DC brought up "the maiden condominium" as an example of the variety of the poems. It is different from the others. I really like the sound in this one. I don't get the full sense. This doesn't turn me off from being intrigued and trying to understand it. These are big words that have never been put in the same line together before in the history of the English language. (Then he reads the whole poem while CG iaughs.rgametocyte" and "banquet" don't rhyme but they go together. "Ga- metocyte" is a sign of life. (CG and MN repeat "maiden condominium" and "control room of the banquet" and wonder what they could be. They are having fun and laughing.) There is some super power going on in this one. And big words. There is wonderful humor in these. Not standup comic humor, thank God. Not one liners. There is comedy in these. Whimsy. Along with lots of seriousness too. A great combination. Definitely one of my favorites. There is no "I" in it, except there is are "eyes"—someone is observing it, thinking it, and feeling it, and commenting about it. It's time of life Issue: powerful, and it's large and small at the same a bird of prey pulls up time; or general and specific at the same time. out of the way into the palm "time of life issue" could be abstract, but "a bird of prey pulls up" (CG says "wow" in the background) is very vivid and specific, and "out of the way into the palm" is both. It has a sort-of opening up. One of the ways good haiku and short poems work best is they look and feel somewhat tight, concentrated, and highlighted and momentary but there is a kind of opening up—and not just a fly-away, not an escape, necessarily, but an opening up. I feel this; this is a fantastic one. -DC I want to sing the praises of this one too. I want the pleasure of saying how much I like it. Because it took me two or three readings before I got it, before I had an image, and then it was transformer time. You know, everything just transformed. This one shows the power of the form because everything is working together, and I just got a strong image. And it changed, too—it wasn't just given to me. I had to work; there was space in the poem for me. I got the connection and that was the 6 the maiden condominium opens its award-winning gametocyte in the control room of the banquet -DC Then DC quickly mentions "day after day" as another example of good humor. "The maiden condominium" is a good example of something InkWell does well that poets have trouble with. InkWell is relentless in trying to find uncommon things to say and ways of saying things. It's not a coincidence that "gametocyte" and "banquet" almost thyme—Ink- day after day Well uses a concept called echoes to populate poems in the man's can with sonic echoes, a sort of musicality. a man can Poetry seems to be one of the tasks Turing and others con- sider central to the idea of the Turing Test. Recall the first example exchange in a fictional exercise of the test: Q: Please write me a sonnet on the subject of the Forth Bridge. A: Count me out on this one. I never could write poetry. The Forth Bridge is iconic and considered a symbol of Scot- land. In his critique of Thring's idea, Geoffrey Jefferson wrote the following in the British Medical Journal [13]: Not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and EFTA_R1_02074944 EFTA02702513 not by the chance fall of symbols, could we agree that machine equals brain—that is, not only write it but know that it had written it. -Jefferson, The Mind of Mechanical Man, 1949 What does InkWell tell us about this? InkWell selects topics to write about, and then chooses a set of personality traits to display, a set of controlling mood words to use to steer what it says about the topic, and overarching subsenses to direct its inner gaze. Indeed InkWell uses randomness as part of its composition strategy, but as DC pointed out, "<InkWell> is not a random person, <InkWell>'s not a random guy." But does InkWell feel these thoughts and emotions? That's basi- cally what the Thring Test is trying to define. Recent work on consciousness (e.g. "The Ego Tunnel" by Thomas Metzinger [141) has something to say about that, but perhaps the best thought is that in writing this, Jefferson mistakes or misun- derstands the poetic / creative process. Writing a poem is not fundamentally an emotional, expres- sive explosion—it's a deliberate task using practiced skills. It's not Walt Whitman's "1 sound my barbaric yawp over the roofs of the world" [15]. The poem "Howl" by Allen Ginsberg [16] (see Appendix) was mythologized as being a performance piece that was recorded and published (this was part of the testimony at the obscenity trial surrounding the poem), but it was written over a period of nearly two years with critical evaluation by friends brought to bear and specific writing techniques explored and exploited. Ginsberg himself com- mented on the intellectually directed choices and investiga- tions he made while creating the poem. InkWell can be thought of as operating deliberately too. Like Ginsberg, InkWell can decide to experiment with long lines; InkWell can decide the degree and nature of deep in the dark— musicality using rhythms and sounds; InklArell can the power of snow decide to make sense or be crazy; and many other walking in the deepness things like this, but all are deliberate artistic choices. Like real poets, InkWell uses skills to create art. Poets who use feelings alone are the best targets for the criticism of "chance fall of symbols." After Inkwell writes a poem. does it know that it had writ- ten it? In a literal sense it does—it records each poem in a log, sometimes (depending on parameters I set) also noting the artistic choices it made. But in the sense Jefferson meant, no. There is no phenomenal self model in play. That is. Inkwell doesn't maintain an internal representation of what it is do- ing aside from representing its artistic choices. What about the question 'Baring imagines: "Please write me a sonnet on the subject of...." Recall that Inkwell can be directed to look at a topic based on a set of words suggested to it. In Spring 20151 was demoing InkWell to a former long- time colleague; he asked me "can you ask it to write a haiku about this: blues guitar and loud music." I asked InkWell to write five poems, and this was one of them: 7 tuned adrenalin my musk, a beat•boogled heedful I believe this is...well, you decide. CG pointed out one that seemed funny—"the powerful head." DC commented on it as follows: Those words are all deadly—potentially deadly. Unpo- etic, right? They're abstract. Who ever has used "cogni- tion" in a poem? There are some world records being set here. After three lines you realize the poem has turned itself upside down—this poem undercuts itself. Maybe because "powerful head" is already the brain or mind, and it's passing the buck to either itself or some sub-brain or sub-mind, but to support cognition, which means it's thinking about passing the buck on thinking. I didn't want to go there. I'm feeling sorry for whoever is caught up in this (meaning the speaker), because it's just the opposite of what it just said. It's "support cogni- tion," but.. thank goodness I didn't quite go there, even though it wants me to all the time. the powerful head designates its powerful head to support cognition -oc "Deep in the dark" is the first poem to catch my attention from the original 2000 InkWell wrote. The great thing about it ("deep in the dark") I like is that the word "dark" of the first line contrasts with the unexpressed "white" of the snow in the second line. The last line puts them together. -MN I see an echo of "stopping by woods." This is a good echo to have. I really do like "the deepness." It res- cues it. I really can't say why but I know. I tried changing it to "depth." But it's a musical thing or an aural thing. Or "depth" is too familiar and conventional. Each line has a "the" and one could play around with removing them. But removing any of them removes also the par- ticularness of the image. "The" slackens the lines—makes them looser—but it also makes them more immediate, familiar, and more specific -oc According to the most extreme form of this view the only way by which one could be sure that machine thinks is to be the machine and to feel oneself thinking. -Turing, computing Machinery and Intelligence. 1950 EFTA_R1_02074945 EFTA02702514 This is the consciousness argument. In its extreme form the only unequivocal way to look at consciousness is solip- sism—it's just me, babe. But 'Raring rejects that and works toward Jefferson's objection about writing a sonnet by consid- ering whether a viva voce would satisfy him—an oral exam in which the interrogator asks detailed questions about the sonnet.' This leads to this interesting question: to what degree does InkWell know about the poems it writes? Inkwell certainly is not programmed to respond to ques- tions such as "why did you use these particular words right here," but it has an accessible representation of the reasons for all its choices. InkWell decides which artistic choices to make, either through whimsy or by reading a text, how much to weigh them against each other, and which moods or out- side influences to consider. These choices are enshrined in a misfit function InkWell constructs—InkWell composes the source code for this function and then compiles it—and all the choices sit in data structures. You might comment, "Ga- briel, you're exaggerating all this," but these explicit traces are my answer to the difficult question, "how do you debug InkWell?" I need to see how and why all the decisions were made, because the only significant bugs arise from domain- related mistakes, which manifest as surprising utterances and never exceptions or type errors. And to figure them out, I need to examine InkWell's state of mind, as it were. And were I so inclined, I could program InkWell to access more gently this self model when quizzed—more gently than by using data structure inspectors and debuggers. "I chose this pair of words because they sparked off each other well with syllable sounds without being blatant rhymes: because I wanted to come off as extraverted while channeling remorse: because I was trying to include a subtext of explo- ration and discovery. They were also very Hemingwayesque. And the best other choices were these, and they just didn't measure up." Inkwell can't say that, but looking at its param- eters, it's sense descriptions, its halos, its musicality settings. its target personality, the writer's ngrams it's trying to mimic, the recorded results of the component factors measured in InkWell's misfit function, etc. for a particular poem. I can trivially report it. One way to look at it is that InkWell has an effective, op- erational self model, but Inkwell itself is not yet in that self model, and thus Inkwell is only partway toward being con- scious. Inkwell modifies its own self model to change how it makes art. When we "talk" to InkWell about these inner changes and factors, we do so in a nonhuman language. and InkWell responds in the same language. Is this ok? Is this enough? . This is calkd the-Pickwick' lest, becauseTuring's essay describes a series of queslions about Charks Dickcns's "The Pickwick Papers." Sec Appendix. 8 Turing wrote: We also wish to allow the possibility that an engineer or team of engineers may construct a machine which works, but whose manner of operation cannot be sat- isfactorily described by its constructors because they have applied a method which is largely experimental. -Turing, Computing Machinery and Intelligence. 1950 Today we read this as referring to machine learning. Some parts of InkWell are machine learned—judgments about per- sonality and emotions in a text, for example. In some cases these judgments are baked into dictionaries and databases. The central part of InkWell is a meta-heuristic optimization process, the basis of whose operation can be explained, but whose detailed operation in any particular instance is a bit mysterious. The construction of the misfit function is sym- bolic, and unpacking how that function directed the result of the optimization is explainable. That bird of prey poem: I felt a lot of doublenesses, and I love doublenesses. I wouldn't describe it as really dark, even though there is darkness in it. I find it also comical—not really funny. There's whimsy to it, a whimsy tone to it, both. This is a form of doubleness—dark and comical / whimsical—and I don't know how you do d— how you, Richard, do it. This is a very large, small poem. It sounds quiet to me. The last line is not threatening, but the poem starts out threatening. Not to the exclu- sion of others, but this one is really terrific. Raring remarks that Lady Lovelace wrote in her memoir the following (17]: The Analytical Engine has no pretensions to originate anything. It can do whatever we know how to order it to perform. -Countess of Lovelace, Translator's Notes.... 1843 This leads to Thring's "surprise" concern: A better variant of the objection says that a machine can never 'take us by surprise.' -Turing, Computing Machinery and Intelligence. 1950 Everyone—including 'TVring—who has ever programmed has been taken by surprise by their code. And not because of bugs and errors—and not because of randomness particularly. For Inkwell it's because of its relentlessness trying to find in- teresting things, such as the phrase in the King James Bible EFTA_R1_02074946 EFTA02702515 scrupulous in the twilight— the price of gold chases the way of the world in power a reasonable assumption— by myself, sampling in chocolate with the best set of sonic similarities to the noun "computer programmer": "provoked me to anger." I can ask InkWell specifically to surprise me because it has ngrams for millions of texts written since 1890 118) including frequencies of appearance, and I can ask for rare or never-seen combinations. I can ask InkWell to search for unusual syn- onyms; I can ask it to write unlike particular writers. When there are dozens of constraint types with both positive and negative weights, there are few limits to surprise. The poets were surprised, too. CG: "'sampling in chocolate'" is surprising language"; "'guitar-shaped coloring' is surpris- ing. It evokes brown / beige because guitars are made of wood, and it's interesting / surprising that a shape could evoke a color." DC: "I don't know ex- actly the sense of this, but i like the surprise. the sound, the sonic surprise of 'scrupulous in the twilight'"; "i recognize all of it as poetry because of the surprisingness of the language." I asked about the use of language in the poems. CG: "Awe- some." DC: "Good noise. Surprising in a good way. a few days— intriguing. Lots of variety. Big, small, long short, y "Ws° • evaluations with Monte-Carlo rollouts. Our uitar-sha browsing g p loud, soft, complex, plain. Not a single track." ed coloring program AlphaGo integrates these components purpose-built software robots like those that play checkers, chess, backgammon, and Go. These are intended to show that human-level expertise in narrow domains can be exhibited or at least simulated. The Jeopardy!-playing Watson is close to these narrow robots because the domain is trivia—the sort of stuff that Google is good at finding. Watson moves closer to the universal notion of thinking that Turing was approaching. Turing ended his essay with an appeal to a learning ap- proach to get machines close to human abilities. As noted learning is generally taken as machine learning these days. As I write this essay, AlphaGo just marched to victory against a very strong human Go player (Lee Se-dol, a 9-dan profes- sional Go player). As David Silver et al wrote [201: We have developed, for the first time, effective move selection and position evaluation functions for Go, based on deep neural networks that are trained by a novel combination of supervised and reinforcement learning. We have introduced a new search algorithm that successfully combines neural network The Turing Test is of course bogus. At least in the form Tur- ing envisioned. A common strategy for passing is to dodge questions. typically using humor and distractions. Turing's own example shows a dodge as an accept- able response; "Count me out on this one. I never could write poetry." Beginning in 2008 a series of practical Turing tests have been conducted under academic scrutiny, run using the best interpretation of Turing's specifications. In June 2014 an extensive set of interrogations were conducted at the Royal Society [19J. This produced ISO parallel tran- scripts, each of which contains a single interrogator posing questions for five minutes to a human and a chatterbot, with the responses being returned side-by-side at the same time on the same screen. In the Appendix you can see a sample par- allel transcript. In this sample the LHS (left-hand side of the screen) was a female adult human, and the RHS was Eugene, a chatterbot. The judge misjudged the LHS to definitely be a machine and the RHS to be a non-native English speaking human. The judge got it backward. The human on the LHS had weak responses while the machine on the RHS tried to dominate the conversation and was definitely more lively than the LHS. The chatterbot pretending to be Eugene Goostman, a 13-year-old Ukrainian boy, was declared to have passed the Turing Test for having fooled more than 30% of the judges. Part of Thring's idea was that the unexpected scope of ques- tions would be the key to deciding whether the computer was thinking sufficiently like a human. This is in contrast with the 9 together, at scale, in a high-performance tree search engine. -Silver et al, Mastering the Game of Go with Deep Neural Networks and Tree Search, 2016 Here the issue of viva vice comes up—how would AlphaGo explain why it made a particular move? Answers of the form "7 is better than 6" won't work well, but perhaps the people who developed AlphaGo can intuit such answers. AlphaGo lost game four, and here is what was reported in the press 1211: According to tweets from DeepMind founder Denies Hassabis, however, this time AlphaGo really did make mistakes. The Al "thought it was doing well, but got confused on move 8Z" Hassabis said, later clarifying that it made a mistake on move 79 but only realized its error by 87. -httpwwww.theverge.com/2016/3/13/11184328/alphago- deepinind-go-rnatch-4-result And Ttiring would seem to respond when he wrote: May not machines carry out something which ought to be described as thinking but which is very different front what a man does? This objection is a very strong one, but at least we can say that if nevertheless, a ma- chine can be constructed to play the imitation game satisfactorily, we need not be troubled by this objection. -Turing, Computing Machinery and Intelligence, 1950 EFTA_R1_02074947 EFTA02702516 An early goal of artificial intelligence was to understand how people think and act. Most Al research from the 1%0s, 1970s, and 1980s was directed toward symbolic Al, which was writing programs whose inner workings—being directed to- ward emulating thought—could be understood and explained. In the 1980s it became apparent that programs that could do the mental work humans do could form the basis for an in- dustry. Around the same time, progress in machine learning was accelerating alongside advances in computer power, and the idea that it was important to understand how Al programs did what they do was swept aside. One could always fall back on the pop idea that human intellectual performance had an intuitive, only faintly understandable side, along with a deliberate, conscious side—that is, a machine learning side and a symbolic side. This is the heart of the Moravec paradox. Perhaps this pop idea has some merit. After I revealed to the poets in my writers workshop that the poems I presented were actually produced by a program, two of them were good with it though they expressed surprise. The third, though. was quite upset. CG said that it was un- fair for me to keep that information secret—because in such workshops it's assumed implicitly that the work is produced by the writers sitting right there, and all the comments are made with that in mind, including that it is proper to be gentle with those comments. This because all the participants are graduates of the same writing program, and hence are linked by a special. "caring" bond. CG said that perhaps critiques could have been more blunt had the rouse not been in place. I countered by saying that a poem boils down to the words on the page, and everything else is contextual interpretation. We can assume that the person believed to have written the poem is important to its effect on a reader, but it all begins with the words on the page—even if "by the chance fall of symbols' [13j. The upshot of CG's comment is that if it's known that a poem was written by a machine. then the poem's inhuman- ity could be explored and perhaps highlighted. CG said that knowing the haiku were written by a computer would open the door for blunt comments. However, the other side of this coin is more interesting: if it's known that a poem was writ- ten by a human, then the poem's humanity can be explored and highlighted. This is the side of the coin CG said was the default for the workshop. But how does one know that a poem was written by a hu- man? By seeing evidence. The entity claiming authorship looks human, acts human. The Turing Test? Could we say that the Turing Test is what makes us human—at least in the eyes of of other people (or other Thring Test passers)? In this case, aside from my claim of authorship,' the evidence of human 2. Actually, my claim was literally truthful: "Ere been working on short poems recently." 10 authorship are the haiku themselves—forming a feedforward loop in the best case. After some evidence is found that the writer is human, the poem is examined more thoroughly, finding even more evidence of this. Recall what CG said: "... it wasn't just given to me. I had to work; there was space in the poem for me. I got the connection and that was the plea- sure for me. The talon isn't mentioned, but you can see it...." •-••••-• Consider surprise and the Lovelace objection—programs do only what we tell them to do and therefore cannot be considered "Turing human." An extreme form of surprise is for the program to do something far removed from its basic programming. InkWell has never created a recipe nor has it proved a difficult theorem. How could it and why would it? You'd need a program designed to survive and thrive in a dynamic environment to discover and achieve novel capa- bilities. To quote the fictional character from "The Martian" I221, "you solve one problem, and you solve the next one, and then the next. And if you solve enough problems, you get to come home." This is the universal version of the evolution- ary and learning objective function—you reward behaviors and ideas that enable you to live. The environment provides opportunities for learning, and simple, built-in mechanisms ratchet that into new capabilities, habits, and proclivities. This is reminiscent of unsupervised and semi-supervised learning. This is finding patterns in data never seen before; there is no reliable way to do this, but when the learner stumbles across some sort of reinforcement, this can frosted winter, bridge black. turn into a weak form of supervised learning. And in ice white the real world, refinement is always possible. All of this leads to Searle's Chinese Room argument 1231 and the role of consciousness in artificial intelligence. Here is a greatly simplified and distilled rephrasing of Searle's issue: there must be something inside of or causally created by the brain that is operating on meaning and intentions, not sym- bols and syntax; this thing is consciousness. He argues this by putting a human (himself, actually) in place of, essentially, the CPU in an Al program that seems to pass a Turing-like Test in Chinese. He remarks that mechanically processing the rules of Chinese question answering does not constitute "understanding" Chinese because he (Searle) playing the part of the CPU wouldn't: ...it seems to me quite obvious...that I do not under- stand a word of the Chinese stories. I have inputs and outputs that are indistinguishablefrom those of the na- tive Chinese speaker,...but I still understand nothing. For the same reasons, Schank's computer understands nothing of any stories, whether in Chinese, English, or whatever, since in the Chinese case the computer is me, and in cases where the computer is not me, the corn- EFTA_R1_02074948 EFTA02702517 puter has nothing more than I have in the case where 1 understand nothing. -Searle. Minds, Brains, and Programs, 1980 Yet somewhere inside or nearby a person there is a thing or a clutch of things such that if you replace it or those with Searle. Searle would understand what the person does solely by doing what it or they do. I believe Searle is talking about consciousness. But Turing argued that relentlessly seeking out conscious- ness in a program is asking more of the program than we normally ask of other people. As CG might say, we afford people the courtesy of believing they are people. Yet the Turing Test seems to be about finding hints someone is at home in(side) the test subject. The Pickwick test, which Turing made up, is about exploring inner thought processes. Jefferson is after this too. But what of programs? InkWell is the program whose hu- manity we are exploring—does it exhibit traits generally asso- ciated with people, especially creative people who are typically thought of as feeling beings? And if that question even makes sense, is this realm of questions part of a reasonable battery of requirements and specifications? We talk about the "ilities" but are these external and to some extent internal traits part of an "ility" we can discuss? The traits describe being able to act like a human mind, and to be able to answer questions about how and why the program performed certain human mindful behaviors—the Pickwick test. To be able to explain a program's mental behavior probably requires some sort of understandable reification of its innards. And this also implies that the innards of a program are im- portant to its humanity or at least to our understanding of its humanity. This means that this kind of programming is not like typical software engineering, where getting the job done with good performance, correctness, and maintain- ability is paramount. k.a..• •• • In the end a group of expert writers and poets believed that eighteen haiku written by Inkwell were worthy of be- ing considered real and sometimes real good poetry. These writers and poets believed I wrote these haiku, and the ques- tion is how much that mattered. One of three poets in my writers' workshop believed this made a difference, and the other two did not. There also seemed to be opportunities for Turing ratchet- ing: finding some human elements in a poem increases the 11 feeling that the poet is a real person, and that increases the likelihood of finding more and deeper human elements in the poem—or at least increases the incentive to look for hints of humanity and the energy to look with. In writing InkWell I was trying to explore how poets and "real" writers write—I was trying to capture in a program what I had learned while studying how to write poetry. I in- tentionally started at the word-choice end of things where many of the effects writers use are hidden in sound (noise). connotation, mood, author personality, and influence. By pursuing this I intentionally made the internals of Inkwell as expressive as was reasonable so I could see the effects of changes and additions, as well as study how the different in- tended effects interacted to produce different texts. Neverthe- less, InkWell has many learned aspects—machine learned at my hands, machine learned by others, and learned through curated and automatically produced dictionaries and data- bases. The interplay between the symbolic and learned aspects of InkWell can be observed decently well, and perhaps the nature of this interplay could provide useful research results into the possible nature of the mind. I have written a poem every day since March 18, 2000. That's a lot of poems. Some nights when I sit down to write my daily poem, I "don't have it," as they say. My talent has taken the night off, nothing happened during the day to serve as a trigger, or I'm simply a little too fatigued to crush it. For the past few years when this happens, I have turned to InkWell to help me. I tell it some odd topics to consider and ask it to write a few dozen poems. And from those I'll revise to a good poem or will use the sequence as a starting place. InkWell is a good helper. The Thring Test provides an interesting lens or instrument useful for exploring what we can make of the semi-living na- ture of programs that are designed for a little bit more than their useful effects. We could go crazy exploring all the ins and outs of philosophy of mind, strong and weak AI, con- sciousness, machine learning versus symbolic deliberation, and intuition versus reasoning. but all I'm wondering about is the lesson to learn from a group of hardcore poets taking Inkwell as a colleague. EFTA_R1_02074949 EFTA02702518 References IIJ Richard P. Gabriel, Jilin Chen, Jeffrey Nichols, InkWell: A Creative Writer's Creative Assistant, Creativity & Cog- nition, 2015. Pi Alan M. Turing, Computing Machinery and Intelligence, Mind, October 1950. (3] Alan M. Thring et al, The Mind and the Computing Ma- chine, The Rutherford Journal. 1947. httpt://en.wikipedia.org/wiki/Haiku Robert Han, editor, "The Essential Haiku: Versions of Bubo, Buson, & Issa," The Ecco Press, 1995. F. Scott Fitzgerald, "The Great Canby," Charles Scrib- ner's Sons, 1925. John Gardner, "The Art of Fiction: Notes on Craft for Young Writers," Vintage. 1991. Arthur Samuel. Some studies in machine learning using the game of checkers, IBM Journal of Research and De- velopment Volume 44, Issue: 1.2. 2000. Garry Kasparov, The Chess Master and the Computer, review of Chess Metaphors: Artificial intelligence and the Human Mind by Diego Rasskin-Gutman, in New York Review of Books, Volume 57, Number 2, Febru- ary 11, 2010. (101 Kenneth Colby, Turing-like Indistinguishability Tests for the Validation of a Computer Simulation of Paranoid Pro- cesses, Artif. Intel]. 3(1-3) 1972. pp. 199-221. HI] Hans Moravec, Mind Children, Harvard University Press, 1988. (12] John Steinbeck, "The Grapes of Wrath," The Viking Press, 1939. (5] (61 (71 (81 (9] 1131 Geoffrey Jefferson, The Mind of Mechanical Man. British Medical Journal, 1949. 1141 Thomas Metzinger, "The Ego Tunnel: The Science of the Mind and the Myth of the Self," Basic Books, 2010. DM Walt Whitman, Song of Myself. "Leaves of Grass," self- published, 1855. (161 Allen Ginsberg, "Howl and Other Poems," City Lights. 1956. (171 Luigi Federico Menabrea, Ada Lovelace, Sketch of the Analytical Engine invented by Charles Babbage...with notes by the translator, translated by Ada Lovelace, in Richard Taylor, "Scientific Memoirs 3," Richard and John E. Taylor. pp. 666-731, 1843. (181 http://storage.googleapis.comThooksMgrams/books/datasetsv2. html 1191 Kevin Warwick & Huma Shah. Can machines think? A report on Turing test experiments at the Royal Society, Journal of Experimental & Theoretical Artificial Intel- ligence. 2015. (20] David Silver et al. Mastering the Game of Go with Deep Neural Networks and Tree Search, Nature 529, pp.484- 489, 28 January 2016. (211 http://www.theverge.com/2016/3/13/11184328/alphago-deep- mind-go-match-4-result (221 The Martian (2015 movie). http://www.imdb.comititlei U36593881 (231 John Searle. Minds, Brains, and Programs, Behavioral and Brain Sciences. 1980. EFTA_R1_02074950 EFTA02702519 Appendix deep in the dark— the power of snow walking in the deepness I'm not the same on an island with destructive rhythm behind a rock on the green slope dead soldiers' spirits the powerful head designates its powerful head to support cognition this grave— no one sees it mortality, mortality shopping parade— people step over the broken cassette a bitch. this deep in trick a fortiori not a man not this fatalist murderousness, deathwatch. but your dead subroutine time of life issue: a bird of prey pulls up out of the way Into the palm awake in the dark— the edge of the water can spread in your presence day after day in the man's can a man can scrupulous in the twilight— the price of gold chases the way of the world in power tuned adrenalin my music. a beat-boogied headful under the sea a fish becomes human In an air pocket a crooked rag day— by myself dunking distracted sardines bare branches. tonight again stars, stars arc misprints Thirty-two Haiku an on-the-far-side summer night— whipping up high tea. [1276911 we stripped pickles 14242311 aboard a boat a round table dancing— (467771) an old song [2665271 shoved off the stairs— falling I become [4833111 a rainbow [359659) the maiden condominium opens its award-winning gametocyte [3660191 in the control room of the banquet 1238801) [4833371 [3571911 [263573) [357781) a reasonable assumption— by myself, sampling in chocolate from the boulder smiling up at heaven the continent begins old lift: through the grille three women in pastel t-shirts rural signal, cannot understand Oregon —agricultural 13635891 1159947) [1733171 13844731 parted in the middle— the authority of the air conditioner (1087911 perfection In the brightness 11356971 too late: the last express passes through [306473) the dust of gardens 14907571 [471853) [3030191 [275309) (2656511 a blue anchor— grains of grit in a tall sky sewing pirates Imitate the ways of ordinary people myself for instance dental hospital— dead files line the light casings the hostile defense leads its problematic rear, the rear of frustration a few days— I361597) [102941] 12592831 13482091 by myself, [494497) browsing guitar-shaped coloring 1160967) (269239) in rags and crystals. sometimes with a shred of sense an odd dignity Haiku labeled with primes were written by Inkwell 13 13498231 EFTA_R1_02074951 EFTA02702520 .. ... 7 m rc cornetist Tri — ... 01 ° atomic-number" m fibonacci-number . i,_ 2_.. I temente scorer i „ d folktale . r preyalenceadsup 4 b_ numerousness ”., complex-quanrinc1/10iinist or-miss u a ,_unselected ognksman w 4- clarinetist lutanist t at, n: C. .... §:violoncellist> .- ›..'6..L. trumpeter. _ 0 ,,, shooter 1 ..o.c >r e a stringer .- e ac o palpebraumplano-player participanO (=Go) prinitcluantity communicatory t !LI 0, 13).—alcard—player sorft-"" e rhyg w = rix O- isoccer- playerwolistom ic-t hockey-player floating-point-numberree. 2 u C fiddler ' ° V,hornist SOD-SION - 47, c-6 billiard-player oxidation-numbet 0, .w iso _ v i vjaavolleyball-playerlutist fl c— r° gam ist to E. recorder-player ,..2 c i_.0 luirawl pan om Clarinettist keyboardist 2most-valuable-player2 E 3-5, jazzim cea-nhol, e 2 football-player. B y- rilayerMUSiCiall shifting r-.it t baseball-player c worii-our accompanist tlergodic nurn rharpist rhythm-and-blues-musicianoymerositym 2 .- 1- payer -player constant ta: d lacrosse-player rock-'n'-roll-musician 2 bowl-- 41 4- t. E ci":3- 71:11 g-xnarra 'vets. rat ig a er e die?' ict esri Ortis t I °>-...- ° bell-ringerws chess-player o. count1,1 anClt iti cSieaend C 0 u ballplayerin k to player E o - grandmaster- sitar-player porous-wove ru — .t ibistguitar -player .E bassooniis$ it ire 6:. harm . oniser= „,. 1,_ oo / Mal er binist 'wore osavxopjf . `0::: -_ :electric-2Ow w IfittJadrate serverwtcod u r ionist._= ,y,6 onist w aim cro to .0 a a , e r i 0 O. 0, — IV -0 ecimal °pianist er an sman 0, or „s o i. cirri t m golfer i=2 1tn nu r muitiplier-facror prim mu =risKer w v, . = re piper soloist fourth-Power.° majority — Appendix — al mvp oboist — ° Appendix Figure 1 Agreeableness 10.0 Conscientiousness -10 Extraversion 0.0 Neuroticism 10.0 Openness 10.0 Openness-To-Change 0.0 Big5 Strength 3.19 Common Word Bonus 2.90 Global Halo Frost.halo Global Halo Bonus 434 Local Halo Bonus 5.76 Local Sense Bonus 40.00 Local Predicates Bonus 938 Synonym Proximity Bonus 4.97 Constraint Bonus 3.41 Rhyme Bonus 47.21 Howl, Allen Ginsberg, first three lines Echo Bonus 37.29 Synonym Search Depth 2 <choose> Search Depth 1 Synonym Decay Rate 0.57 Maximum Synonym Sense,. 2 2-gram-bonus 75 3-gram-bonus 85 4-gram-bonus 85 5-gram-bonus 75 Writer-2-gram-bonus 27.698803 Writer-3-gram-bonus 45.65159 Writer-4-gram-bonus 71.91725 Writer-5-gram-bonus 44.505795 Writer File Poemsrpg.txt Long Words True Appendix Table I I saw the best minds of my generation destroyed by madness, starving hysterical naked. dragging themselves through the negro streets at dawn looking for an angry fix. angelheaded hipsters burning for the ancient heavenly connection to the starry dynamo in the machinery of night.... 14 EFTA_R1_02074952 EFTA02702521 Royal Society Turing Test, June 2014 Transcript I -LHS (16:20:48) Judge what isyour worst memory in your life so far? (16:21:06) Entity: I don't understand. Explain. (16:21:35) Judge: memory of something that happened to you that was bad or painful (16:22:581 Entity: Hmm that a tricky question. My life is won- derful. I overcome many bad things that have happened to me in the past. However, the most painful memory I have had is performing very poorly in my GCSE's when I was younger. (16:23:561 Judge: did you fail? (16:24:12) Entity: Partially. (16:24:35) Judge: explain? (16:25:04) Entity: Some grades turned out to be good. Some turned out to be quite bad. (16:25:37) Judge: and that is the worst thing that has happened to you so far? Mmm how old are you? The Pickwick Test Transcript I—RHS (16:21:00) Judge: what did you have for breakfast today (16:21:10) Entity: Is it 'the third degree'? I'm not going to answer such a provocative question! Well, let's go on though. (16:21:56( fudge: do go on...your call? (16:22:06) Entity: It is highly possible. By the way, what's your occupation? I mean—could you tell me about your work? (16:22:30( Judge: so you are asking me now eh? I am an actress and writer (16:22:381 Entity: Well, I'll read something of your books. if they ain't too tedious (which is highly possible:-) (16:23:151 ►udge: grammer...aren't too tedious! A bit rude of you... (16:23:17) Entity: Calm down, please. (16:23:49) Judge: i asure you i am very calm ... are you? (16:24:01( Entity: Don't even bother me with the fact that you are very calm ... are mine. And I forgot to ask you where you are from ... (16:25:01) Judge: I am from Dorset...where are you from? (16:25:08) Entity: I am from Ukraine, from the city called Odessa. You might have heard about it. Interrogator. In the first line of your sonnet which reads "Shall I compare thee to a summer's day," would not "a spring day" do as well or better? Witness: It wouldn't scan. Interrogator: How about "a winter's day." That would scan all right. Witness: Yes, but nobody wants to be compared to a winter's day. Interrogator: Would you say Mr. Pickwick reminded you of Christmas? Witness: In a way. Interrogator. Yet Christmas is a winter's day. and I do not think Mr. Pickwick would mind the comparison. Witness: I don't think you're serious. By a winter's day one means a typical winter's day, rather than a special one like Christmas. t5 EFTA_R1_02074953 EFTA02702522