Turing test
{{dablink|For the ''Doctor Who'' novel named after the test, see [[The Turing Test (novel)]].
For the opera named after the test, see under the composer, [[Julian Wagstaff]].}}

The '''Turing test''' is a proposal for a test of a [[machine]]'s capability to demonstrate intelligence. Described by [[Alan Turing]] in the 1950 paper "[[Computing Machinery and Intelligence]]," it proceeds as follows: a human judge engages in a natural language conversation with  one human and one machine, each of which try to appear human; if the judge cannot reliably tell which is which, then the machine is said to pass the test.  In order to test the machine's intelligence rather than its ability to render words into audio, the conversation is limited to a text-only channel such as a computer keyboard and screen (Turing originally suggested a [[Teleprinter|teletype machine]], one of the few text-only communication systems available in 1950).

[[Image:Turing Test version 3.png|thumb|The "standard interpretation" of the Turing Test, in which player C, the interrogator, is tasked with trying to determine which player - A or B - is a computer and which is a human. The interrogator is limited to only using the responses to written questions in order to make the determination]]

==History==

While the field of [[artificial intelligence]] is said to have been founded in 1956,<ref>{{Harvnb|Crevier|1993|pp=47-49}}, {{Harvnb|Russell|Norvig|2003|p=17}} and {{Harvnb|Copeland|2003|p=1}}</ref> its roots extend back considerably further. The question as to whether or not it is possible for machines to think has a long history, firmly entrenched in the distinction between [[Dualism (philosophy of mind)|dualist]] and [[materialism|materialist]] views of the mind. From the perspective of dualism, the [[mind]] is [[Non-physical entity|non-physical]] (or, at the very least, has [[Property dualism|non-physical properties]]<ref>For an example of [[Property dualism]], see [[Qualia]].</ref>), and therefore cannot be explained in purely physical terms. The materialist perspective, on the other hand, argues that the mind can be explained physically, and thus leaves open the possibility of minds that are artificially produced.<ref>Noting that materialism does not ''necessitate'' the possibility of artificial minds (for example, [[Roger Penrose#Physics and consciousness|Roger Penrose]]), any more than dualism necessarily ''precludes'' the possibility (for example, [[Property dualism]]).</ref>

===Alan Turing===

In more practical terms, researchers in Britain had been exploring "machine intelligence" for up to ten years prior to 1956. Alan Turing in particular had been tackling the notion of machine intelligence since at least 1941,<ref>{{Harvnb|Copeland|2003|p=1}}</ref> and one of the earliest known mentions of "computer intelligence" was made by Turing in 1947.<ref>{{Harvnb|Copeland|2003|p=2}}</ref> In Turing's report, "Intelligent Machinery", he investigated "the question of whether or not it is possible for machinery to show intelligent behaviour",<ref>{{Harvnb|Turing|1948|p=412}}</ref> and as part of that investigation proposed what may be considered the forerunner to his later tests:

{{quote|"It is not difficult to devise a paper machine which will play a not very bad game of chess. Now get three men as subjects for the experiment. A, B and C. A and C are to be rather poor chess players, B is the operator who works the paper machine. ... Two rooms are used with some arrangement for communicating moves, and a game is played between C and either A or the paper machine. C may find it quite difficult to tell which he is playing."|{{Harvnb|Turing|1948|p=431}} }}

Thus by the time Turing published "Computing Machinery and Intelligence", he had been considering the possibility of machine intelligence for many years. This, however, was the first published paper<ref>"Intelligent Machinery" was not published by Turing, and didn't see publication until 1968 in Evans, C. R. & Robertson, A. D. J. (1968) ''Cybernetics: Key Papers'', University Park Press.</ref> by Turing to focus exclusively on the notion.

Turing began his 1950 paper with the claim: "I propose to consider the question, 'Can machines think?'"<ref name="T433">{{Harvnb|Turing|1950|p=433}}</ref> As Turing highlighted, the traditional approach to such a question is to start with definitions, defining both the terms [[machine]] and [[intelligence]]. Nevertheless, Turing chose not to do so. Instead he replaced the question with a new question, "which is closely related to it and is expressed in relatively unambiguous words".<ref name=T433/> In essence, Turing proposed to change the question from "Do machines think?" into "Can machines do what we (as thinking entities) can do?"<ref>{{Harvnb|Harnad|p=1}}</ref> The advantage of the new question, Turing argued, was that it "drew a fairly sharp line between the physical and intellectual capacities of a man.<ref name=1950Turing434>{{Harvnb|Turing|1950|p=434}}</ref>

To demonstrate this approach, Turing proposed a test that was inspired by a [[party game]] known as the "Imitation Game", in which a man and a woman go into separate rooms, and guests try to tell them apart by writing a series of questions and reading the typewritten answers sent back.  In this game, both the man and the woman aim to convince the guests that they are the other. Turing proposed recreating the imitation game as follows:  

{{quote|"We now ask the question, 'What will happen when a machine takes the part of A in this game?' Will the interrogator decide wrongly as often when  the game is played like this as he does when the game is played between  a man and a woman? These questions replace our original, 'Can machines  think?'"|{{Harvnb|Turing|1950|p=434}}}}

Later in the paper he suggested an "equivalent" alternative formulation involving a judge conversing only with a computer and a man.<ref>{{Harvnb|Turing|1950|p=446}}</ref>

While neither of these two formulations precisely match the version of the Turing Test that is more generally known today, a third version was proposed by Turing in 1952. In this version, which Turing discussed in a [[BBC]] radio broadcast, Turing proposes a jury which asks questions of a computer, and where the role of the computer is to make a significant proportion of the jury believe that it is really a man.<ref>{{Harvnb|Turing|1952|pp=524-525}}. Turing does not seem to distinguish between "man" as a gender and "man" as a human. In the former case this formulation would be closer to the Imitation Game, while in the later it would be closer to current depictions of the test.</ref>

Turing's paper considered nine common objections, which include all the major arguments against artificial intelligence that have been raised in the years since his paper was first published. (See ''[[Computing Machinery and Intelligence]]''.)<ref>{{Harvnb|Turing|1950}} and see {{Harvnb|Russell|Norvig|2003|p=948}} where they comment "Turing examined a wide variety of possible objections to the possibility of intelligent machines, including virtually all of those that have been raised in the half century since his paper appeared."</ref>

===ELIZA, PARRY and the Chinese room===

Blay Whitby lists four major turning points in the history of the Turing Test: the publication of "Computing Machinery and Intelligence" in 1950; the announcement of [[Joseph Weizenbaum]]'s [[ELIZA]] in 1966; Kenneth Colby's creation of [[PARRY]], which was first described in 1972; and the Turing Colloquium in 1990.<ref>{{Harvnb|Whitby|1996|p=53}}</ref> 

ELIZA works by examining a user's typed comments for keywords. If a word is found a rule is applied which transforms the user's comments, and the resulting sentence is then returned. If a keyword is not found, ELIZA responds with either a generic response or by repeating one of the earlier comments.<ref>{{Harvnb|Weizenbaum|1966|p=37}}</ref> In addition, Weizenbaum developed ELIZA to replicate the behavior of a [[Person-centered psychotherapy|Rogerian psychotherapist]], allowing ELIZA to be "free to assume the pose of knowing almost nothing of the real world."<ref name=Weizenbaum42>{{Harvnb|Weizenbaum|1966|p=42}}</ref> Due to these techniques, Weizenbaum's program was able to fool some people into believing that they were talking to a real person, with some subjects being "very hard to convince that ELIZA ... is ''not'' human."<ref name=Weizenbaum42 /> Thus ELIZA is claimed by many to be one of the programs (perhaps the first) that are able to pass the Turing Test.<ref>{{Harvnb|Thomas|1995|p=112}}</ref><ref name=Weizenbaum42 /> 

Colby's PARRY has been described as "ELIZA with attitude"<ref>{{Harvnb|Bowden|2006|p=370}}</ref> - it attempts to model the behavior of a [[Paranoia|paranoid]] [[Schizophrenic|schizophrenic]], using a similar (if more advanced) approach to that employed by Weizenbaum. In order to help validate the work, PARRY was tested in the early 1970s using a variation of the Turing Test. A group of experienced psychiatrists analyzed a combination of real patients and computers running PARRY through [[teletype]] machines. Another group of 33 psychiatrists were shown transcripts of the conversations. The two groups were then asked to identify which of the "patients" were human, and which were computer programs.<ref>{{Harvnb|Coby|Hilf|Weber|Kraemer|1972|p=42}}</ref> The psychiatrists were only able to make the correct identification 48% of the time - a figure consistent with random guessing.<ref>{{Harvnb|Saygin|Cicekli|Akman|2000|p=501}}</ref>

While neither ELIZA nor PARRY were able to pass a strict Turing Test, they - and software like them - suggested that software might be written that was able to do so. More importantly, they suggested that such software might involve little more than databases and the application of simple rules. This led to [[John Searle]]'s 1980 paper, "Minds, Brains, and Programs", in which he proposed an argument against the Turing Test. Searle described a [[thought experiment]] known as the [[Chinese room]] that highlighted what he saw as a fundamental misinterpretation of what the Turing Test could and could not prove: while software such as ELIZA might be able to pass the Turing Test, they might do so by simply manipulating symbols of which they have no understanding. And without understanding, they could not be described as "thinking" in the same sense people do. Searle concludes that the Turing Test can not prove that a machine can think, contrary to Turing's original proposal.<ref>{{Harvnb|Searle|1980}}</ref>

Arguments such as that proposed by Searle and others working in the [[philosophy of mind]] sparked off a more intense debate about the nature of intelligence, the possibility of intelligent machines and the value of the Turing test that continued through the 1980s and 1990s.<ref>{{Harvnb|Saygin|Cicekli|Akman|2000|p=479}}</ref>

===1990s and beyond===

1990 was the 40th anniversary of the first publication of Turing's "Computing Machinery and Intelligence" paper, and thus saw renewed interest in the test. Two significant events occurred in that year. The first with the Turing Colloquium, which was held at the [[University of Sussex]] in April, and brought together academics and researchers from a wide variety of disciplines to discuss the Turing Test in terms of its past, present and future. The second significant event was the formation of the annual [[Loebner prize]] competition.

The Loebner prize was instigated by [[Hugh Loebner]] under the auspices of the Cambridge Center for Behavioral Studies of [[Massachusetts]], [[United States]], with the first competition held in November, 1991.<ref>{{Harvnb|Sundman|2003}}</ref> As Loebner describes it, the competition was created to advance the state of AI research, at least in part because while the Turing Test had been discussed for many years, "no one had taken steps to implement it."<ref>{{Harvnb|Loebner|1994}}</ref> The Loebner prize has three awards: the first prize of $100,000 and a gold medal, to be awarded to the first program that passes the "unrestricted" Turing test; the second prize of $25,000, to be awarded to the first program that passes the "restricted" version of the test; and a sum of $2000 (now $3000) to the "most human-like" program that was entered each year. [[As of 2007]], neither the first nor second prizes have been awarded.

The running of the Loebner prize led to renewed discussion of both the viability of the Turing Test and the aim of developing artificial intelligences that could pass it. ''[[The Economist]]'', in an article entitled "Artificial Stupidity", commented that the winning entry from the first Loebner prize won, at least in part, because it was able to "imitate human typing errors".<ref>{{harvnb|"Artificial Stupidity"|1992}}</ref> (Turing had considered the possibility that computers could be identified by their ''lack'' of errors, and had suggested that the computers should be programmed to add errors into their output, so as to be better "players" of the game).<ref>{{harv|Turing|1950|page=448}}</ref> The issue that ''The Economist'' raised was one that was already well established in the literature: perhaps we don't really ''need'' the types of computers that could pass the Turing Test, and perhaps trying to pass the Turing Test is nothing more than a distraction from more fruitful lines of research.<ref>{{harvnb|Shieber|1994|page=77}}</ref> Equally, a second issue became apparent - by providing rules which restricted the abilities of the interrogators to ask questions, and by using comparatively "unsophisticated" interrogators, the Turing Test can be passed through the use of "trickery" rather than intelligence.<ref>{{harvnb|Shapiro|1992|page=10-11}} and {{harvnb|Shieber|1994}}, amongst others</ref>

==Versions of the Turing test==

[[Image:The Imitation Game.png|thumb|The imitation game, as described by Alan Turing in "Computing Machinery and Intelligence". Player C, through a series of written questions, attempts to determine which of the two players is a man, and which of the two is the woman. Player A - the man - tries to trick player C into making the wrong decision, while player B tries to help player C]]
There are at least three primary versions of the Turing test - two offered by Turing in "Computing Machinery and Intelligence" and one which Saul Traiger describes as the "Standard Interpretation".<ref name=Traiger2000>{{harvnb|Traiger|2000}}</ref> While there is some debate as to whether or not the "Standard Interpretation" is described by Turing or is, instead, based on a misreading of his paper, these three versions are not regarded as being equivalent,<ref name=Traiger2000 /> and are seen as having different strengths and weaknesses. As [[empirical]] tests they conform to a proposal published in 1936 by [[A J Ayer]] on how to distinguish between a conscious man and an unconscious machine. In his book ''[[Language, Truth and Logic]]'' Ayer states that 'The only ground I can have for asserting that an object which appears to be conscious is not really a conscious being, but only a dummy or a machine, is that it fails to satisfy one of the empirical tests by which the presence or absence of consciousness is determined'.<ref>[[Language, Truth and Logic]] (p140), Penguin 2001</ref> 

===The imitation game===

Turing described a simple party game which involves three players. Player A is a man, player B is a woman, and player C (who plays the role of the interrogator) can be of either gender. In the imitation game, player C - the interrogator - is unable to see either player A or player B, and can only communicate with them through written notes. By asking questions of player A and player B, player C tries to determine which of the two is the man, and which of the two is the woman. Player A's role is to trick the interrogator into making the wrong decision, while player B attempts to assist the interrogator.<ref>{{harvnb|Turing|1950|p=433-434}}</ref>

In what Sterret refers to as the "Original Imitation Game Test",<ref name=Moor2003 /> Turing proposed that the role of player A be replaced with a computer. The computer's task is therefore to pretend to be a woman and to attempt to trick the interrogator into making an incorrect evaluation. The success of the computer is determined by comparing the outcome of the game when player A is a computer against the outcome when player A is a man. If, as Turing puts it, "the interrogator decide[s] wrongly as often when the game is played [with the computer] as he does when the game is played between a man and a woman"<ref name=1950Turing434 />, then it can be argued that the computer is intelligent.

[[Image:Turing Test Version 1.png|thumb|The Original Imitation Game Test, in which the player A is replaced with a computer. The computer is now tasked with the role of pretending to be a woman, while player B continues to attempt to assist the interrogator]]

The second version comes later in Turing's 1950 paper. As with the Original Imitation Game Test, the role of player A is performed by a computer. The difference is that now the role of player B is to be performed by a man, rather than by a woman. 

{{quote|"Let us fix our attention on one particular digital computer ''C''. Is it true that by modifying this computer to have an adequate storage, suitably increasing its speed of action, and providing it with an appropriate programme, ''C'' can be made to play satisfactorily the part of A in the imitation game, the part of B being taken by a man?"|{{Harvnb|Turing|1950|p=442}}}}

In this version both player A (the computer) and player B are trying to trick the interrogator into making an incorrect decision.<ref name=Saygin252>{{Harvnb|Saygin et al|2000|p=252}}</ref>

===The standard interpretation===

A common understanding of the Turing test is that the purpose was not specifically to test if a computer is able to fool an interrogator into believing that it is a woman, but to test whether or not a computer could ''imitate'' a human.<ref name=Saygin252 /> While there is some dispute as to whether or not this interpretation was intended by Turing (for example, Sterrett believes that it was,<ref name="Moor2003">{{Harvnb|Moor|2003}}</ref> and thus conflates the second version with this one, while others, such as Traiger, do not<ref name="Traiger2000" />), this has nevertheless led to what can be viewed as the "standard interpretation". In this version, player A is a computer, and player B is a person of either gender. The role of the interrogator is not to determine which is male and which is female, but to determine which is a computer and which is a human.<ref>{{harvnb|Traiger|2000|p=99}}</ref>

===Imitation game vs. standard Turing test===

There has been some controversy over which of the alternative formulations of the test Turing intended.<ref name=Moor2003 /> Sterret argues that two distinct tests can be extracted from Turing's 1950 paper, and that, ''pace'' Turing's remark, they are not equivalent. The test that employs the party game and compares frequencies of success in the game is referred to as the "Original Imitation Game Test" whereas the test consisting of a human judge conversing with a human and a machine is referred to as the "Standard Turing Test", noting that Sterret equates this with the "standard interpretation" rather than the second version of the imitation game.  Sterrett agrees that the Standard Turing Test (STT) has the problems its critics cite, but argues that, in contrast, the Original Imitation Game Test (OIG Test) so defined is immune to many of them, due to a crucial difference:  the OIG Test, unlike the STT, does not make similarity to a human performance the criterion of the test, even though it employs a human performance in setting a criterion for machine intelligence.  A man can fail the OIG Test, but it is argued that this is a virtue of a test of intelligence if failure indicates a lack of resourcefulness.  It is argued that the OIG Test requires the resourcefulness associated with intelligence and not merely "simulation of human conversational behaviour".  The general structure of the OIG Test could even be used with nonverbal versions of imitation games.<ref>{{Harvnb|Sterrett|2000}}</ref>

Still other writers<ref>{{Harvnb|Genova|1994}}, {{Harvnb|Hayes|Ford|1995}}, {{Harvnb|Heil|1998}}, {{Harvnb|Dreyfus|1979}}</ref> have interpreted Turing to be proposing that the imitation game itself is the test, without specifying how to take into account Turing's statement that the test he proposed using the party version of the imitation game is based upon a criterion of comparative frequency of success in that imitation game, rather than a capacity to succeed at one round of the game.

===Should the interrogator know about the computer?===

Turing never makes it clear as to whether or not the interrogator in his tests is aware that one of the participants is a computer. To return to the Original Imitation Game, Turing states only that Player A is to be replaced with a machine, not that player C is to be made aware of this replacement.<ref name=1950Turing434 /> When Colby, Hilf, Weber and Kramer tested PARRY, they did so by assuming that the interrogators did not need to know that one or more of those being interviewed was a computer during the interrogation.<ref>{{harvnb|Colby et al|1972}}</ref> But, as Saygin and others highlight, this makes a big difference to the implementation and outcome of the test.<ref>{{harvnb|Saygin et al|2000|p=60}}</ref>

==Strengths of the test ==

The power of the Turing test derives from the fact that it is possible to talk about anything. Turing wrote "the question and answer method seems to be suitable for introducing almost any one of the fields of human endeavor that we wish to include."<ref>{{Harvnb|Turing|1950}} under "Critique of the New Problem"</ref> [[John Haugeland]] adds that "understanding the words is not enough; you have to understand the ''topic'' as well."<ref>{{Harvnb|Haugeland|1985|p=8}}</ref>

In order to pass a well designed Turing test, the machine would have to use [[natural language processing|natural language]], to [[commonsense reasoning|reason]], to have [[knowledge representation|knowledge]] and to [[machine learning|learn]]. The test can be extended to include video input, as well as a "hatch" through which objects can be passed, and this would force the machine to demonstrate the skill of [[computer vision|vision]] and [[robotics]] as well. Together these represent almost all the major problems of [[artificial intelligence]].<ref>[[Stuart J. Russell]] and [[Peter Norvig]] write "These six disciplines represent most of AI". {{Harvnb|Russell|Norvig|2003|p=3}}</ref>

==Weaknesses of the test ==
The test has been criticized on several grounds.

===Human intelligence vs. intelligence in general===
[[Image:Weakness of Turing test 1.svg|thumb]]
The test is explicitly [[anthropomorphic]]. It only tests if the subject ''resembles'' a human being. It will fail to test for intelligence under two circumstances:
* It tests for many behaviors that we may not consider intelligent, such as the susceptibility to insults or the temptation to lie. A machine may very well be intelligent without being able to chat ''exactly'' like a human.
* It fails to capture the ''general'' properties of intelligence, such as the ability to solve difficult problems or come up with original insights. If a machine can solve a difficult problem that no person could solve, it would, in principle, fail the test.

[[Stuart J. Russell]] and [[Peter Norvig]] argue that the anthropomorphism of the test prevents it from being truly useful for the task of engineering intelligent machines. They write: "Aeronautical engineering texts do not define the goal of their field as 'making machines that fly so exactly like pigeons that they can fool other pigeons.'"<ref name=RussellNorvig3>{{Harvnb|Russell|Norvig|2003|p=3}}</ref>

The test is also vulnerable to naivete on the part of the test subjects. If the testers have little experience with [[chatterbot]]s they may be more likely to judge a computer program to be responding coherently than someone who is aware of the various tricks that chatterbots use, such as changing the subject or answering a question with another question. Such tricks may be misinterpreted as "playfulness" and therefore evidence of a human participant by uninformed testers, especially during brief sessions in which a chatterbot's inherent repetitiveness does not have a chance to become evident.

===Real intelligence vs. simulated intelligence===

The test is also explicitly [[behaviorist]] or [[functionalist]]: it only tests how the subject ''acts.'' 

A machine passing the Turing test may be able to ''simulate human conversational behaviour'' but the machine might just follow some cleverly devised rules. Two famous examples of this line of argument against the Turing test are [[John Searle]]'s [[Chinese room]] argument and [[Ned Block]]'s [[Blockhead (computer system)|Blockhead]] argument.

Even if the Turing test is a good operational definition of intelligence, it may not indicate that the machine has [[consciousness]], or that it has [[intentionality]]. Perhaps intelligence and consciousness, for example, are such that neither one necessarily implies the other. In that case, the Turing test might fail to capture one of the key differences between intelligent machines and intelligent people.

== Predictions and tests ==
Turing predicted that machines would eventually be able to pass the test. In fact, he estimated that by the year 2000, machines with 10<sup>9</sup> [[bit]]s (about 119.2 [[mebibyte|MiB]]) of memory would be able to fool 30% of human judges during a 5-minute test.  He also predicted that people would then no longer consider the phrase "thinking machine" contradictory. He further predicted that [[machine learning]] would be an important part of building powerful machines, a claim which is considered to be plausible by contemporary researchers in [[Artificial intelligence]].

By extrapolating an [[Technological singularity#Accelerating change|exponential growth]] of technology over several decades, [[Future Studies|futurist]] [[Ray Kurzweil]] predicted that Turing-test-capable computers would be manufactured around the year 2020, roughly speaking.<ref>{{Harvnb|Kurzweil|1990}}</ref>  See the [[Moore's Law]] article and the references therein for discussions of the plausibility of this argument.

[[As of 2008]], no computer has passed the Turing test as such. Simple conversational programs such as [[ELIZA]] have fooled people into believing they are talking to another human being, such as in an informal experiment termed [[AOLiza]]. However, such "successes" are not the same as a Turing Test. Most obviously, the human party in the conversation has no reason to suspect they are talking to anything other than a human, whereas in a real Turing test the questioner is actively trying to determine the nature of the entity they are chatting with. Documented cases are usually in environments such as [[Internet Relay Chat]] where conversation is sometimes stilted and meaningless, and in which no understanding of a conversation is necessary. Additionally, many internet relay chat participants use English as a second or third language, thus making it even more likely that they would assume that an unintelligent comment by the conversational program is simply something they have misunderstood, and do not recognize the very non-human errors they make. See [[ELIZA effect]].

The [[Loebner prize]] is an annual competition to determine the best Turing test competitors.  Although they award an annual prize for the computer system that, in the judges' opinions, demonstrates the "most human" conversational behaviour (with learning AI [[Jabberwacky]] winning in [[2005]] and [[2006]], and [[Artificial Linguistic Internet Computer Entity|A.L.I.C.E.]] before that), they have an additional prize for a system that in their opinion passes a Turing test. This second prize has not yet been awarded. The creators of Jabberwacky have proposed a personal Turing Test: the ability to pass the imitation test while attempting to specifically imitate the human player, with whom the AI will have conversed at length before the test.<ref>See http://www.jabberwacky.com/s/PTT100605.pdf</ref>

In [[2008]] the competition for the [[Loebner prize]] is being co-organised by [[Kevin Warwick]] and held at the [[University of Reading]] on [[October 12]]. The directive for the competition is to stay as close as possible to Turing's original statements made in his 1950 paper, such that it can be ascertained if any machines are presently close to 'passing the test'. An academic meeting discussing the Turing Test, organised by the [[Society for the Study of Artificial Intelligence and the Simulation of Behaviour]], is being held in parallel at the same venue.  

Trying to pass the Turing test in its full generality is not, as of 2005, an active focus of much mainstream academic or commercial effort.  Current research in AI-related fields is aimed at more modest and specific goals.  

The first bet of the [[Long Bet Project]] is a [[United States dollar|$]]10,000 one between [[Mitch Kapor]] (pessimist) and [[Ray Kurzweil]] (optimist) about whether a computer will pass a Turing Test by the year [[2029]]. The bet specifies the conditions in some detail.<ref>http://www.longbets.org/1#terms</ref>

==Variations of the Turing test==

A modification of the Turing test, where the objective or one or more of the roles have been reversed between computers and humans, is termed a [[reverse Turing test]].

Another variation of the Turing test is described as the [[Subject matter expert Turing test]] where a computer's response cannot be distinguished from an expert in a given field.

As brain and body scanning techniques improve it may also be possible to replicate the essential [[data element]]s of a person to a computer system.{{Fact|date=March 2007}} The [[Immortality test]] variation of the Turing test would determine if a person's essential character is reproduced with enough fidelity to make it impossible to distinguish a reproduction of a person from the original person.

The [[Minimum Intelligent Signal Test]] proposed by [[Chris McKinstry]], is another variation of Turing's test, but where only binary responses are permitted.  It is typically used to gather statistical data against which the performance of [[artificial intelligence]] programs may be measured.

Another variation of the reverse Turing test is implied in the work of psychoanalyst Wilfred Bion,<ref>{{Harvnb|Bion|1979}}</ref> who was particularly fascinated by the "storm" that resulted from the encounter of one mind by another. Carrying this idea forward, R. D. Hinshelwood<ref> {{Harvnb|Hinshelwood|2001}}</ref> described the mind as a "mind recognizing apparatus", noting that this might be some sort of "supplement" to the Turing test. To make this more explicit, the challenge would be for the computer to be able to determine if it were interacting with a human or another computer. This is an extension of the original question Turing was attempting to answer, but would, perhaps, be a high enough standard to define a machine that could "think" in a way we typically define as characteristically human.

Another variation is the Meta Turing test, in which the subject being tested (for example a computer) is classified as intelligent if it itself has created something that the subject itself wants to test for intelligence.

==Practical applications==
[[Stuart J. Russell]] and [[Peter Norvig]] note that "AI researchers have devoted little attention to passing the Turing Test",<ref name=RussellNorvig3> since there are easier ways to test their programs: by giving them a task directly, rather than through the roundabout method of first posing a question in a chat room populated with machines and people. [[Alan Turing]] never intended his test to be ''used'' as a real, day-to-day measure of the intelligence of AI programs. He wanted to provide a clear and understandable example to help us discuss the [[philosophy of artificial intelligence]].<ref>{{Harvnb|Turing|1950}} under The Imitation Game, where he writes "Instead of attempting such a definition I shall replace the question by another, which is closely related to it and is expressed in relatively unambiguous words."</ref>

Real Turing tests, such as the [[Loebner prize]], do not usually force programs to demonstrate the full range of intelligence and are reserved for testing [[chatterbot]] programs. However, even in this limited form these tests are still very rigorous. The 2008 [[Loebner prize]] however is sticking closely to Turing's original concepts - for example conversations will be for 5 minutes only. 

[[CAPTCHA]] is a form of [[reverse Turing test]]. Before being allowed to do some action on a [[website]], the user is presented with alphanumerical characters in a distorted graphic image and asked to recognise it. This is intended to prevent automated systems from abusing the site. The rationale is that software sufficiently sophisticated to read the distorted image accurately does not exist (or is not available to the average user), so any system able to do so is likely to be a human being.

== In popular culture ==
 
In the ''[[Dilbert]]'' comic strip on Sunday [[30 March]] [[2008]],<ref>[http://www.dilbert.com/strips/comic/2008-03-30/ Dilbert strip for Mar 30, 2008]</ref>, Dilbert says, "The security audit accidentally locked all of the developers out of the system", and his boss responds with only meaningless, [[tautology (rhetoric)|tautological]] [[thought-terminating cliché]]s, "Well, it is what it is." Dilbert asks "How does that help" and his boss responds with another cliche, "You don't know what you don't know."  Dilbert replies, "Congratulations. You're the first human to fail the Turing Test."  For that day, "turing test" was the 43<sup>rd</sup> most popular [[Google]] search.<ref>[http://google.com/trends/hottrends?sa=X&date=2008-3-30 Google Trends: Mar 30, 2008<!-- Bot generated title -->]</ref>

The character of [[Ghostwheel]] in [[Roger Zelazny]]'s [[The Chronicles of Amber]] is mentioned to be capable of passing the Turing Test.

The webcomic [[xkcd]] has referred to Turing and the Turing test.<ref>http://xkcd.com/329/</ref>

[[Rick Deckard]],in the movie [[Blade Runner]], used a Turing Test to determine if Rachael was a [[Replicant]].

==See also==
<div style="-moz-column-count:3; -webkit-column-count:3; column-count:3;">
* [[Artificial intelligence in fiction]]
* [[Graphics Turing Test]]
* [[HAL 9000]] (Kubrick's AI)
* [[Mark V Shaney]] (USENET bot)
* [[Simulated reality]]
* [[Technological singularity]]
* [[Uncanny valley]]
* [[Voight-Kampff machine]]
</div>

==Notes==
{{reflist|2}}

==References==
{{Refbegin|2}}

* {{Citation | title = Artificial Stupidity | year = 1992 | date = 1992-09-01 | journal = [[The Economist]] | volume = 324 | issue = 7770 | pages = 14}}
* {{Citation | last=Bion | first=W.S. | year=1979 | chapter=Making the best of a bad job | title=Clinical Seminars and Four Papers | publisher=Abingdon: Fleetwood Press. }}
* {{Citation | last = Bowden | first = Margaret A. | year = 2006| title = Mind As Machine: A History of Cognitive Science | publisher = [[Oxford University Press]] | ISBN = 9780199241446}}
* {{Citation | last = Colby | first = K. M. | last2 = Hilf | first2 = F. D. | last3 = Weber | first3 = S. | last4 = Kraemer | fisrt4 = H. | year = 1972 | title = Turing-like indistinguishability tests for the validation of a computer simulation of paranoid processes | journal = [[Artificial Intelligence (journal)|Artificial Intelligence]] | volume = 3 | pages = 199-221}}
* {{Citation | last = Copeland | first = Jack | authorlink = Jack Copeland | year = 2003 | title = The Turing Test | work = The Turing Test: The Elusive Standard of Artificial Intelligence | editor-last = Moor | editor-first = James | publisher = Springer | ISBN = 1-40-201205-5}} 
* {{Citation  | last = Crevier | first = Daniel | authorlink = Daniel Crevier | year = 1993 | title = AI: The Tumultuous Search for Artificial Intelligence | location = New York, NY | publisher = BasicBooks | ISBN = 0-465-02997-3}}
* {{Citation | last = Dreyfus | first = Hubert  | title = What Computers ''Still'' Can't Do | year =1979 | publisher = MIT Press | location = New York | authorlink = Hubert Dreyfus|isbn= ISBN 0-06-090613-8}}
* {{Citation | last=Genova | first=J. | year=1994 | title=Turing's Sexual Guessing Game | journal=Social Epistemology | volume=8 | number=4 | pages=314-326 | issn=0269-1728}}
* {{Citation| last=Harnad | first=Stevan | author-link=Stevan Harnad | year=2004 | contribution=The Annotation Game: On Turing (1950) on Computing, Machinery, and Intelligence | url=http://cogprints.org/3322/ | editor-last=Epstein | editor-first=Robert | editor2-last=Peters | editor2-first=Grace | title=The Turing Test Sourcebook: Philosophical and Methodological Issues in the Quest for the Thinking Computer | publisher=Klewer }}
* {{Citation |  last = Haugeland | first = John |year = 1985 | title = Artificial Intelligence: The Very Idea | publisher=MIT Press| publisher-place= Cambridge, Mass. | author-link = John Haugeland}}.
* {{Citation | last=Hayes | first=Patrick | last2=Ford | first2=Kenneth | year=1995 | title=Turing Test Considered Harmful | journal=Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (IJCAI95-1),  Montreal, Quebec, Canada. |  pages=972-997}}
* {{Citation| last=Heil | first=John | year=1998 | title=Philosophy of Mind:  A Contemporary Introduction | location=London and New York | publisher=Routledge | isbn=0-415-13060-3 }}
* {{Citation | last=Hinshelwood | first=R.D. | year=2001 | title=Group Mentality and Having a Mind: Reflections on Bion's work on groups and on psychosis }}
* {{Citation | last=Kurzweil | first=Ray | author-link=Ray Kurzweil | year=1990 | title=The Age of Intelligent Machines | isbn=0-262-61079-5 }}
* {{Citation | last = Loebner | first = Hugh Gene | authorlink = Hugh Loebner | title = In response | year = 1994 | journal = [[Communications of the ACM]] | volume = 37 | issue = 6 | pages = 79-82 | url = http://loebner.net/Prizef/In-response.html | accessdate = 2008-03-22}}
* {{Citation| editor-last=Moor | editor-first=James | year=2003 | title=The Turing Test: The Elusive Standard of Artificial Intelligence | isbn=1-4020-1205-5 }}
* {{Citation | last=Penrose | first=Roger | author-link=Roger Penrose | year=1989 | title= [[The Emperor's New Mind|The Emperor's New Mind: Concerning Computers, Minds, and The Laws of Physics]]|publisher= Oxford University Press| isbn=0-14-014534-6 }}
* {{Citation | last = Russell | first = Stuart J. | authorlink = Stuart J. Russell | last2 = Norvig | first2 = Peter | authorlink2 = Peter Norvig | year = 2003 | title = [[Artificial Intelligence: A Modern Approach]] | edition = 2nd | location = Upper Saddle River, NJ | publisher = Prentice Hall | ISBN = 0-13-790395-2}}
* {{Citation | last = Saygin | first = Ayse Pinar | last2 = Cicekli | first2 = Ilyas | last3 = Akman | first3 = Varol | year = 2000 | title = Turing Test: 50 Years Later}} in {{citation | title = The Turing Test: The Elusive Standard of Artificial Intelligence | editor-last = Moor | editor-first = James | year = 2003 | publisher = Springer | ISBN = 1-40-201205-5}}
* {{Citation| last=Searle| first=John | author-link=John Searle | year=1980 | url = http://members.aol.com/NeoNoetics/MindsBrainsPrograms.html  | title = Minds, Brains and Programs | journal = Behavioral and Brain Sciences | volume = 3| issue = 3| pages= 417-457}}. ''Page numbers above refer to a standard [[pdf]] print of the article. See also Searle's [http://www.bbsonline.org/Preprints/OldArchive/bbs.searle2.html original draft].'' 
* {{Citation | last = Shapiro | first = Stuart C. | title = The Turing Test and the economist | year = 1992 | journal = ACM SIGART Bulletin | volume = 3 | issue = 4 | pages = 10-11}}
* {{Citation | last = Shieber | first = Stuart M. | title = Lessons from a Restricted Turing Test | year = 1994 | journal = [[Communications of the ACM]] | volume = 37 | issue = 6 | pages = 70-78 | url = http://www.eecs.harvard.edu/shieber/Biblio/Papers/loebner-rev-html/loebner-rev-html.html | accessdate = 2008-03-25}}
* {{Citation | last=Sterrett | first=S. G. | year=2000 | title=Turing's Two Test of Intelligence | journal=Minds and Machines | volume=10 | number=4 | ISSN= 0924-6495}} (reprinted in The Turing Test: The Elusive Standard of Artificial Intelligence edited by James H. Moor, Kluwer Academic 2003) ISBN 1-4020-1205-5
* {{Citation | last = Sundman | first = John | title = Artificial stupidity | year = 2003 | journal = [[Salon.com]] | date = February 26, 2003 | url = http://dir.salon.com/story/tech/feature/2003/02/26/loebner_part_one/index.html | accessdate = 2008-03-22}}
* {{Citation | last = Thomas | first = Peter J. | year = 1995 | title = The Social and Interactional Dimensions of Human-Computer Interfaces | publisher = [[Cambridge University Press]] | ISBN = 052145302X }}
* {{Citation | last = Traiger | first = Saul | year = 2000 | title= Making the Right Identification in the Turing Test| journal=Minds and Machines | volume=10 | number=4 | ISSN= 0924-6495}} (reprinted in The Turing Test: The Elusive Standard of Artificial Intelligence edited by James H. Moor, Kluwer Academic 2003) ISBN 1-4020-1205-5
* {{Citation | last = Turing | first = Alan | authorlink=Alan Turing | year=1948 | chapter=Machine Intelligence | title = The Essential Turing: The ideas that gave birth to the computer age | editor=Copeland, B. Jack | ISBN = 0-19-825080-0 }}
* {{Citation | last = Turing | first = Alan | authorlink=Alan Turing | year=1950 | title = [[Computing machinery and intelligence]] | journal=Mind | issn=0026-4423 | volume = LIX | issue = 236 |date=October 1950 | pages= 433-460 | url =http://loebner.net/Prizef/TuringArticle.html | doi=10.1093/mind/LIX.236.433}}
* {{Citation | last = Turing | first = Alan | authorlink=Alan Turing | year=1952 | chapter = Can Automatic Calculating Machines be Said to Think? |  title = The Essential Turing: The ideas that gave birth to the computer age | editor=Copeland, B. Jack | ISBN = 0-19-825080-0 }}
* {{Citation | last=Zylberberg | first=A. | last2=Calot | first2=E. | year=2007 | title=Optimizing Lies in State Oriented Domains based on Genetic Algorithms | journal= Proceedings VI Ibero-American Symposium on Software Engineering |  pages=11-18 | ISBN = 978-9972-2885-1-7 }}
* {{Citation | last = Weizenbaum | first = Joseph | authorlink = Joseph Weizenbaum | year = 1966 | title = ELIZA - A Computer Program For the Study of Natural Language Communication Between Man And Machine | journal = [[Communications of the ACM]] | volume = 9| issue = 1 | date = January, 1966 | pages = 36-45 }}
* {{Citation | last = Whitby | first = Blay | year = 1996 | contribution = The Turing Test: AI's Biggest Blind Alley? | title = Machines and Thought: The Legacy of Alan Turing | volume = 1 | editor = Millican, Peter & Clark, Andy | publisher = [[Oxford University Press]] | pages = 53-62 | ISBN = 0-19-823876-2 }}
* {{Citation | last = Adams | first = Scott | year = 2008 | title = Dilbert | Distributed by = [[UFS, inc.]] | url =http://www.dilbert.com/comics/dilbert/archive/images/dilbert2008033349280.jpg}}
{{Refend}}

==Further reading==
{{Refbegin}}
* B. Jack Copeland, ed., ''The Essential Turing: The ideas that gave birth to the computer age'' (2004). ISBN 0-19-825080-0
* [[Larry Gonick]], ''The Cartoon Guide to the Computer'' ([[1983]], originally ''The Cartoon Guide to Computer Science'').  ISBN 0-06-273097-5.
* S. G. Sterrett "Nested Algorithms and the 'Original Imitation Game Test'," Minds and Machines (2002). ISSN 0924-6495
* A.P. Saygin, I. Cicekli, and V Akman (2000), 'Turing Test: 50 Years Later', Minds and Machines 10(4): 463-518. (reprinted in The Turing Test: The Elusive Standard of Artificial Intelligence edited by James H. Moor, Kluwer Academic 2003) ISBN 1-4020-1205-5. (Thorough review. Online version at [http://crl.ucsd.edu/~saygin/papers/MMTT.pdf] )
* Saygin, A.P. & Cicekli I (2002): [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6VCW-461X2JT-7&_user=10&_coverDate=03%2F31%2F2002&_rdoc=1&_orig=search&_sort=d&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=6855fb2d15953bb3ebf30026d4b10f1d Pragmatics in human-computer conversations] (Abstract and links to pdf, if permitted), ''Journal of Pragmatics'', Volume 34, Issue 3, March 2002, Pages 227-258.
{{Refend}}

==External links==
* [http://www.zompist.com/turing.html Twelve reasons to toss the Turing Test]
* [http://www.turingtestopera.com The Turing Test - an Opera by Julian Wagstaff]
* {{dmoz|Computers/Artificial_Intelligence/Natural_Language/Turing_Test|Turing Test}}
* [http://www.rmcybernetics.com/science/cybernetics/ai_turing_test.htm The Turing Test]- How accurate could the turing test really be?
* [http://plato.stanford.edu Stanford Encyclopedia of Philosophy] entry on [http://plato.stanford.edu/entries/turing-test the Turing test], by G. Oppy and [http://www.csse.monash.edu.au/~dld D. Dowe].
* [http://cogsci.ucsd.edu/~asaygin/tt/ttest.html The Turing Test Page] lists recent articles, links, and other info on the test.
* [http://crl.ucsd.edu/~saygin/papers/MMTT.pdf Turing Test: 50 Years Later] reviews a half-century of work on the Turing Test, from the vantage point of 2000.
* [http://www.longbets.org/1 Bet between Kapor and Kurzweil], including detailed justifications of their respective positions.
* [http://www.cogs.susx.ac.uk/users/blayw/tt.html Why The Turing Test is AI's Biggest Blind Alley] by Blay Witby
* [http://www.twinkiesproject.com/turing.html A humorous look] at proving the non-intelligence of a [[Twinkie]]
* [http://www.turinghub.com TuringHub.com] Take the Turing Test, live, online
* [http://www.jabberwacky.com Jabberwacky.com] An AI [[chatterbot]] that learns from and imitates humans
* New York Times essays on machine intelligence [http://www.rci.rutgers.edu/~cfs/472_html/Intro/NYT_Intro/History/MachineIntelligence2.html part 1] and [http://www.rci.rutgers.edu/~cfs/472_html/Intro/NYT_Intro/History/MachineIntelligence2.html part 2]
* [http://ken-jennings.com/blog/?p=102 Today’s Bewildering Conversation with a Three-Year-Old] Jeopardy winner [[Ken Jennings]] blogs about a humorous Turing-challenged conversation with his toddler son.
* [http://vvi.onstreammedia.com/cgi-bin/visearch?user=pbs-saf&template=template.html&query=turing&category=0&viKeyword=turing&submit=Search Machines Who Think"]: [[Scientific American Frontiers]] video on "the first ever [restricted] Turing test."
* [http://www.vhumanforum.com/ Virtual Humans Forum].

[[Category:Turing tests| ]]
[[Category:Philosophy of artificial intelligence]]
[[Category:Alan Turing]]
[[Category:Human-computer interaction]]
[[Category:History of artificial intelligence]]

[[ca:Test de Turing]]
[[cs:Turingův test]]
[[da:Turing-test]]
[[de:Turing-Test]]
[[et:Turingi test]]
[[es:Prueba de Turing]]
[[eo:Testo de Turing]]
[[fa:آزمون تورینگ]]
[[fr:Test de Turing]]
[[gl:Proba de Turing]]
[[ko:튜링 테스트]]
[[it:Test di Turing]]
[[he:מבחן טיורינג]]
[[lv:Tjūringa tests]]
[[hu:Turing-teszt]]
[[nl:Turing-test]]
[[ja:チューリング・テスト]]
[[no:Turingtest]]
[[pl:Test Turinga]]
[[pt:Teste de Turing]]
[[ro:Testul Turing]]
[[ru:Тест Тьюринга]]
[[simple:Turing test]]
[[sk:Turingov test]]
[[sr:Тјурингов тест]]
[[fi:Turingin testi]]
[[sv:Turingtest]]
[[th:การทดสอบของทัวริง]]
[[tr:Turing Testi]]
[[uk:Тест Тьюрінга]]
[[zh:圖靈試驗]]