1. Introduction
In his famous October 1950 paper in Mind, “Computing Machinery and Intelligence”, Turing described an imitation game involving a “question and answer method” and proposed that success in this game provided a “criterion for ‘thinking’” in machines (1950, 435, 436).Footnote 1 Turing’s paper was a dramatic contrast to other articles in the journal in 1950 and the years immediately prior; the few papers that might be described as on philosophy of mind typically discussed the epistemology of sense data, selves as particulars or substances, mental acts as dispositions, psycho-physical parallelism, and souls—nothing on learning machines or arguments for or against the possibility of artificial intelligence (AI). Perhaps for this reason, some early replies to Turing’s paper did not take his proposal very seriously. Only a few months later, Leonard Pinsky whimsically proposed an alternative test for intelligence in machines. He suggested that one of Turing’s computers be sent, as a “Christmas gift”, a “subscription to MIND, retroactive to October, 1950” (Pinsky Reference Pinsky1951, 397–98). The computer, on reading Turing’s article, might realize that “it is thinking about the possibility of machines thinking” and consequently suffer “a nervous breakdown”—if the computer could be cured by the reassurance that it was “only tussling with a pseudo-problem”, this really would demonstrate that a machine thinks, Pinsky declared (1951, 398).
Sober treatments of Turing’s proposed test appeared quite quickly. On the one hand, in April 1952, Wolfe Mays claimed that Turing was a “mechanical necromancer”, in whose account “the meaning of the word ‘thinking’ has changed to such an extent that it has little in common with what we normally mean by it”; Turing’s “behaviouristic” approach failed to see that what is important is not “what” the machine does, “but how it does it”, Mays claimed (Reference Mays1952, 150–53, 158). Several early discussions pressed Mays’ first point; for example, Michael Scriven claimed in 1953 that the expression “intelligent” was “an improper adjective for the noun ‘computer’”, and so talk of intelligent computers implied “a change of meaning in one of these terms” (Reference Scriven1953, 236). On the other hand, in 1951 Donald MacKay said that there is no distinction in principle between the “observable behaviour of a human brain” and the “behaviour possible in a suitably designed artefact”, and in 1952 Henry Cohen stated that arguments against the possibility of thinking machines “lead inevitably, as Turing has shown, to a naïve solipsism” (MacKay Reference MacKay1951, 120; Cohen Reference Cohen1952, 206). Subsequently, F. H. George claimed that Mays had made “no impression whatsoever on Turing’s position”—and, echoing Turing’s sentiments in Mind, that “much dispute” over the capabilities of machines was due merely to “feelings of an emotional kind” (Reference George1956, 248; Reference George1958, 59).Footnote 2
Three-quarters of a century later, the verdict on Turing’s test is often cavalier. For example, an article in October 2025 in Nature claimed that large language models (LLMs) “killed the Turing test”: “[c]hatbots now ace the mathematician’s famous imitation game”, it proclaimed (Gibney Reference Gibney2025). The evidence cited was a 2025 study stating that two LLMs “pass the Turing test” (Jones and Bergen Reference Jones and Bergen2025).Footnote 3 Several recent studies making the same claim have prompted commentators to declare that the test is obsolete—either because we no longer need an existence proof of the possibility of AI, or (assuming these LLMs do not think) because their success in the test counterexamples Turing’s claim to have provided a criterion for thinking in machines. The vital question here, however, is whether these machines have passed Turing’s test, or merely some imitation test (Copeland Reference Copeland2000, Reference Copeland2004a).
Our first aim in the present paper is, taking Turing’s descriptions of his test, to establish definitively his conceptual design for the test. We argue that influential interpretations of his design are incompatible with Turing’s own words; these readings also conflict with the early perception and historical context of his ideas.Footnote 4 A consequence is that any claim that Turing’s test has been passed by unintelligent LLMs, and his criterion for thinking thereby rebutted, is false. Our second aim is to present the results of implementing Turing’s conceptual design for the first time (full technical specifications of our study, data, and analyses are available in Temtsin et al. Reference Temtsin, Proudfoot, Kaber and Bartneck2025). The results of our study do not support a “general consensus”, announced at the October 2025 Royal Society celebration of the Turing test’s 75th anniversary, that AI has passed the test.Footnote 5 When we tested a generative pre-trained transformer (GPT) machine, the machine’s performance was extremely poor.
The paper is in two parts. The first part (sections 2–4) aims to identify Turing’s conceptual design for his test on the basis of exegetical, historical, and experimental considerations.Footnote 6 From Turing’s descriptions, we extract informal specifications of the structure, scoring method, and essential features of his imitation game. The second part (section 5) presents (informally) our new empirical study implementing Turing’s design and the philosophical lessons to be derived from it. The paper, we argue, generates a challenge for AI experimenters: to earn the cachet of passing Turing’s test, their machine must do well in the imitation game presented here.
1.1 Two preliminary points
Discussions of the Turing test often begin with some claim concerning Turing’s philosophical commitments—roughly, either to behaviourism, some classical or non-classical version of computationalism, or (more recently) response-dependence concerning judgements of mind. Our aim is to identify Turing’s conceptual design without tying it to any specific theory of mind, in order that his blueprint may be useful to theorists, regardless of their interpretations of Turing’s philosophy. (We turn briefly to Turing’s philosophy of mind at the beginning of section 5.) For the same reason, we make no assumptions regarding the nature of the concept of intelligence, or of the property of being intelligent, for which success in Turing’s game is to suffice.
In addition, many discussions deny that Turing intended his test as an actual experiment to be performed. For example, at the Royal Society celebration, it was stated that Turing was only “kidding” in his 1950 paper, and that we should not take the paper “too seriously”.Footnote 7 It has been claimed that Turing intended his test merely as a “joke” (Minsky Reference Minsky2013), a “philosophical conversation-stopper” (Dennett Reference Dennett and Michael1985), or as a rhetorical device in order to counter popular objections to the possibility of AI (Sloman Reference Sloman, Barry Cooper and van Leeuwen2013). However, these claims neglect the fact that Turing discussed his test, not only in the Mind paper, but also in a 1952 radio symposium on machine learning, “Can Automatic Calculating Machines Be Said to Think?”, and he anticipated his computer-imitates-human game in “Intelligent Machinery”, an official 1948 report for the National Physical Laboratory (NPL) (Turing et al. Reference Turing, Braithwaite, Jefferson and Newman1952; Turing Reference Turing1948; Copeland Reference Copeland2004a). In addition, these claims overlook the historical context: in the late 1940s and early 1950s, the emergence of the world’s first electronic stored-program computers in the UK and the US had generated vigorous public and scientific debate on the possibility of AI—Turing was aware that a criterion of intelligence in machines was badly needed.Footnote 8
Some commentators claim that Turing intended his test only as a thought experiment (e.g., Mitchell Reference Mitchell2024). However, depicting Turing as solely an armchair philosopher would be almost as inaccurate as characterizing him as solely a pure mathematician. Turing was interested in turning his abstract universal computing machine into concrete machines; at the NPL, for example, Turing and his engineering team were designing his Automatic Computing Engine (Copeland Reference Copeland2004b). In addition, he was interested in actual human reactions to computers. In his NPL report, he described a “little experiment” in which a human (C) plays chess with another human or with a “paper machine”—which is to say, a human being simulating a computer by following a set of rules, using pencil and paper, the only sort of programmable computer then available (1948, 431). C, Turing said, “may find it quite difficult to tell which he is playing” (1948, 431). This experiment anticipated in important respects the imitation game he announced two years later, and Turing ended his description of the chess game with the words, “This is a rather idealized form of an experiment I have actually done” (1948, 431).
2. Turing’s man-imitates-woman game: Rival interpretations
In the second paragraph of his 1950 paper, Turing introduces “a game which we call the ‘imitation game’”, as follows:
It is played with three people, a man (A), a woman (B), and an interrogator (C) who may be of either sex. The interrogator stays in a room apart from the other two. The object of the game for the interrogator is to determine which of the other two is the man and which is the woman. He knows them by labels X and Y, and at the end of the game he says either “X is A and Y is B” or “X is B and Y is A”. (1950, 433)Footnote 9
After more discussion of this man-imitates-woman game, Turing said:
We now ask the question, “What will happen when a machine takes the part of A in this game?” Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? (1950, 434)
Later in the paper Turing suggests that “the question, ‘Can machines think?’ should be replaced by ‘Are there imaginable digital computers which would do well in the imitation game?’” (1950, 442). When a computer is a player, Turing says, the computer takes “the part of A in the imitation game, the part of B being taken by a man” (1950, 442).
A persistent and sometimes torturous debate has grown up around these remarks. In moving from a man-imitates-woman game to a new game involving a computer, is Turing proposing that, in the latter, (1) the interviewees are a computer and a woman, and the computer’s task is to imitate a woman? Or is it that (2) the interviewees are a computer and a man (i.e., male or masculine human being), and the computer’s task is to imitate a man? Or is it that (3) the interviewees are a computer and a human being (of any gender), and the computer’s task is to imitate a human being? All three interpretations can be found in the recent scholarly literature as well as in popular discussions of Turing’s test.
In an influential paper criticizing Turing’s test as “actively harmful”, Patrick Hayes and Kenneth Ford propose (1)—the computer-imitates-woman interpretation.Footnote 10 They say, “If we take [Turing] at his word, the test is rather clever”:
It has a woman and a machine each trying to convince the judge that they are a woman, and the judge’s task is still to decide which is the woman and which, therefore, is not. But this judge is not thinking about the differences between women and machines, but between women and men. The hypothesis that one of his subjects is not human is not even in his natural space of initial possibilities. This judge has exactly the same problem to solve as a judge in the original imitation game and could be expected to bring the same attitudes and skills to the problem. We will call this the gender test. (Hayes and Ford Reference Hayes, Ford and Chris1995, 972)
The conception of Turing’s test as a “gender test” is still prevalent (e.g., Sterrett Reference Sterrett2020; Gonçalves Reference Gonçalves2024).Footnote 11 However, Turing nowhere explicitly mentioned a computer’s imitating a woman. Nor did any of his 1952 co-symposiasts or the early 1950s’ theorists responding to the Mind paper—the philosophers and scientists who could be expected to have most familiarity with Turing’s ideas.
An influential proponent of (2), the computer-imitates-man interpretation, is Judith Genova. According to Genova, Turing held that “being a man is a matter of simulating (imitating, voguing) masculinity” and he “wanted to show the similarity of the two issues”, gender and intelligence (Reference Genova1994, 317). Genova claims that “[t]extual considerations alone” argue for this reading (Reference Genova1994, 314).
Certainly, as quoted above, when a computer takes the role of A, the role of B is to be taken by “a man”. However, at issue is whether Turing’s use here of the word “man” is the “masculine” or the “generic” use; the latter use of the terms “man” and “he”, to indicate a human being irrespective of sex, was common in Turing’s time and did not come under sustained attack until the early 1970s (Curzan Reference Curzan2003; Earp Reference Earp2012). It is evident that Turing used masculine terms generically in the Mind paper: after stating that the interrogator “may be of either sex”, he used throughout the pronoun “he” to refer to the interrogator. In Mind he also remarks, for example: “Thinking is a function of man’s immortal soul”; “God cannot make a man to have no soul”; and “the only way to know that a man thinks is to be that particular man” (1950, 443, 446).Footnote 12 Turing could expect his readers to have no difficulty in identifying the generic use, since they too commonly employed it. For example, Geoffrey Jefferson’s Lister Oration, to which Turing responds in his 1950 paper, was titled “The Mind of Mechanical Man”; in this paper, Jefferson discussed “man’s characteristic desire for knowledge”, “the qualities of man’s mind”, and “the difference between man and beasts” (Reference Jefferson1949, 1105, 1106). The 1950s’ commentaries on Turing’s paper also used “man” as a generic term, and Mays illustrated the generic use when he remarked:
From the point of view of logic there is little to choose (except in precision) between the performance of a man and a machine. A human being might well be looked at as a machine for producing logical conclusions. (Reference Mays1952, 158)
In Turing’s paper, textual considerations argue against, rather than for, the computer-imitates-man interpretation. Most uses of the term “man” in the paper are generic. Turing’s few masculine uses are signalled by a contrast with “woman” (e.g., “God has given an immortal soul to every man and woman” [1950, 443]). When Turing said that the part of B is to be taken by “a man”, the contrast is with “digital computer”—this offers no reason to read the use of “man” as masculine.
So, Turing’s own words, along with those of his milieu and readership, point away from both (1) and (2) and toward (3), the computer-imitates-human interpretation. This fits, as Copeland points out, with Turing’s claim, in his 1951 radio lecture “Can Digital Computers Think?”, that the point of the test was to determine if the computer can “imitate a brain” (Turing Reference Turing1951, 485; Copeland Reference Copeland2004, 436). Also, it is supported by early 1950s’ commentaries; for example, Pinsky said that the object of the imitation game is for the interrogator to determine “which of the participants is a human being and which a machine” (1951, 397).
Yet, it might be inquired: if the computer-imitates-human reading is correct, why did Turing introduce the man-imitates-woman game? Turing’s biographer, Andrew Hodges, offers one answer:
The whole point of this game was that a successful imitation of a woman’s responses by a man would not prove anything. Gender depended on facts which were not reducible to sequences of symbols. In contrast, he [Turing] wished to argue that such an imitation principle did apply to “thinking” or “intelligence”. If a computer, on the basis of its written replies to questions, could not be distinguished from a human respondent, then “fair play” would oblige one to say that it must be “thinking”. (2014, 523)
Hodges’ account, however, faces difficulties. Why would Turing choose a game where successful performance fails as a criterion as a model for a game where successful performance is to succeed as a criterion? Hodges provides no explanation. Nor does he provide any reason why Turing would make this choice without saying anything to make the “whole point” clear. On this, Hodges remarks only that the beginning of Turing’s Mind paper was “one of the few passages of the paper that was not expressed with perfect lucidity” (Reference Hodges2014, 523).
Amy Kind offers another explanation. In her view, it is “critical” that Turing begins with “a sex analogy” (Reference Kind, Maitra and McWeeny2022, 63). According to Kind, Turing had a “mid-twentieth-century conception” of sex, such that it is “both binary and fixed”, and he thought of intelligence too as an “all-or-nothing property”, which encourages a “humancentric prejudice” (2022, 60, 63). However, Turing did not deny that intelligence may exist on a continuum; for example, he wrote of the “real brains” of non-human animals (1951, 483). Nor does the fact that the man-imitates-woman game presents a binary choice imply that (or that Turing believed that) sexual identity is binary—any more than a red/green colour blindness test implies that being red is a mind-independent, discrete property.
A simpler explanation of Turing’s man-imitates-woman game is that it was part of his response to the general public’s and scientific community’s antipathy to the possibility of AI.Footnote 13 If Turing were to have any hope of persuading his audience of the reasonableness of his test, he needed to smooth its path. A man’s imitating a woman was a familiar sight in mid-twentieth century Britain, in theatres and on BBC radio and television. According to The Times, female impersonation held “a deep and irresistible appeal” and was “a universal theme” bringing “multitudes” to watch.Footnote 14 The Cambridge Footlights and the Cambridge Amateur Dramatic Club regularly had female impersonators; the “very presence of men in female parts” in these productions, The Times said, generated a “feeling of lightheartedness”.Footnote 15 In the late 1940s and early 1950s, an analogy with a man-imitates-woman game was a perfect vehicle to break the ice for Turing’s radical proposal.
3. Scoring the computer-imitates-human game: Three competing frameworks
In Turing’s Mind paper, “do[ing] well” in the computer-imitates-human game is a sufficient condition of intelligence or thinking in machines (1950, 436)—but what constitutes doing well? Three influential answers are given.
Numerous commentaries suppose that Turing’s framework for scoring the game is to be found in this remark (P):
I believe that in about fifty years’ time it will be possible to programme computers, with a storage capacity of about 109, to make them play the imitation game so well that an average interrogator will not have more than 70 per cent. chance of making the right identification after five minutes of questioning. (Turing Reference Turing1950, 442)
Using this framework, a computer passes the test if it attains a “30 per cent or more deception rate” in “a five-minute interaction” between the interrogator and the two interviewees, the interrogator’s time shared roughly equally between the interviewees (Shah and Warwick Reference Shah and Warwick2010, 452). These are the “strict Turing rules”, advocates of this framework claim (Warwick and Shah Reference Warwick and Shah2016a, 990).
However, (P) plainly expresses a prediction (Copeland Reference Copeland2000; Moor Reference Moor2001). Immediately before (P), Turing says that he will “explain first my own beliefs in the matter”; following (P), he states additional beliefs about the future of AI, ending with the words “I believe further that no useful purpose is served by concealing these beliefs” (1950, 442). A prediction may stipulate a scoring protocol, but it need not: for example, my prediction that a student will score no higher than 30% in their philosophy exam does not imply that 30% is the pass mark. Is there reason to think that (P) states both a prediction and the framework for scoring the game?Footnote 16 Not, at least, in the early responses to Turing’s paper, which did not read (P) in this way. Of this part of Turing’s text, Mays said only: “Turing is firmly confident that in fifty years time it will be possible to programme computors to play the imitation game” (Reference Mays1952, 148). In addition, concerning Turing’s scoring framework, Pinsky said only: “If the interrogator cannot distinguish the machine from the human player in a high percentage of the attempts, presumably machines do think” (1951, 397).
Moreover, there are reasons to think that (P) does not state Turing’s framework for scoring the game. In his 1952 broadcast, Turing appeared to set the bar high for any machine, saying that in his test a “considerable proportion” of interrogators must be fooled—and even the advocates of the “strict Turing rules” acknowledge that a 30% pass mark is “weak” (Turing et al. Reference Turing, Braithwaite, Jefferson and Newman1952, 495; Shah and Warwick Reference Shah and Warwick2010, 452). This aside, treating (P) as stating Turing’s scoring framework generates a discrepancy between his words in the 1950 paper and those in his 1952 broadcast. In the latter, when Max Newman suggested that “it will be a long time from now” before a machine in Turing’s test “is to stand any chance with no questions barred”, Turing replied, “Oh yes, at least 100 years, I should say” (Turing et al. Reference Turing, Braithwaite, Jefferson and Newman1952, 495). If (P) states Turing’s authorized scoring framework, there is a prima facie inconsistency between the “about fifty years” prediction and the “at least 100 years” prediction. To resolve this inconsistency, advocates of the “strict Turing rules” declare that it is “unlikely” that Turing “truly felt” that a machine would pass the test (i.e., would attain a 30% deception rate in five minutes) by the end of the twentieth century (Warwick and Shah Reference Warwick and Shah2016b, 39). However, this is to claim, implausibly, that Turing did not mean what he very clearly said.
The second scoring framework proposed as Turing’s own (and often used in recent LLM studies) centres on chance. For example, James Moor holds that a computer does well if, after the game is played by “many different interrogators”, each with “ample opportunity to ask many questions”, the “probability of the average interrogator correctly identifying the machine is not significantly greater than 50 percent” (Reference Moor1976, 249–50). Other accounts of Turing’s test require that the computer must do better than chance—that is, the probability of the average interrogator’s judging correctly must be less than 50%. However, Turing does not indicate either formulation in the Mind paper or in his other comments on his test. An additional problem with several of the proposed chance frameworks is that they also stipulate that the game is to last only five minutes. However, except for (P), Turing did not mention a duration for the imitation game, or even state that the game has a set duration, in his 1950 paper or anywhere else.
A third well-known interpretation of Turing’s scoring framework is due to Jack Copeland (Copeland Reference Copeland2004a, Reference Copeland2023). In Copeland’s view, Turing’s remark “Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman?” indicates “the protocol for scoring the test”: a computer passes the test if the interrogator in the computer-imitates-human game does no better than the interrogator in the man-imitates-woman game (Copeland Reference Copeland2004a, 436). Using this framework, Turing’s discussion of the man-imitates-woman game does not lack in “lucidity”, as Hodges claimed; on the contrary, Turing meant exactly what he said.
Copeland does not address the question of why Turing might have chosen this framework. It might seem odd: although comparing the performances of different test-takers on the same test is common scientific practice, Copeland’s interpretation appears to compare the performances of different test-takers (the interrogators) on different tests (the computer-imitates-human game and the man-imitates-woman game). This oddity may disappear, though, if both games are regarded simply as two tokens of one type of test, an imitation test—Turing speaks of the machine “being put through one of my imitation tests” (Turing et al. Reference Turing, Braithwaite, Jefferson and Newman1952, 503). Different test-takers are then being compared on the same type of test. Certainly, this only invites another question: why select performance in an “imitation test” as a criterion of thinking in machines? This question, however, concerns Turing’s philosophical justification for his test, rather than his design, and so arises for any proposed scoring framework.
In sum, of the three prominent scoring frameworks for the computer-imitates-human game proposed as Turing’s pick, his own words exclude the first (30% deception in five minutes), neither support nor exclude the second (chance)—so long as it does not incorporate a five-minute limit—and are in line with the third (comparison with the man-imitates-woman game). Juxtaposing Moor’s proposed framework with Copeland’s, the former appears to have the greater explanatory burden. It has to explain why Turing failed to mention a chance benchmark and also to provide an alternative construal of Turing’s words, “Will the interrogator decide wrongly …”.
4. Essential features of Turing’s design
Additional key aspects of Turing’s design are commonly overlooked or misunderstood.
4.1 Controlling for anthropomorphism
Frequently, including in many online “Turing tests”, Turing’s game is presented as a two-player game, where the interrogator must decide whether a single hidden interviewee is a computer or a human being. Yet, as quoted above, in Mind Turing stated that “a game which we call the ‘imitation game’” is “played with three people”. Where one interviewee is a computer, the three-player setup also has theoretical advantages.
Turing himself pointed out a flaw in the two-player game. In his 1952 broadcast, he described what appears to be a two-player imitation game, saying that interrogators should be required to judge “quite a number of times”, where “sometimes they really are dealing with a man and not a machine”—this is to “prevent them saying ‘It must be a machine’ every time without proper consideration” (Turing et al. Reference Turing, Braithwaite, Jefferson and Newman1952, 495). Turing’s observation is astute: a defect in the two-player game is that judges, to avoid the embarrassment of being fooled by a machine, might always or mostly declare the interviewee to be a computer. Such behaviour has been observed in two-player imitation games held as part of the Loebner Prize Contests in Artificial Intelligence. This flaw does not exist in the three-player imitation game, since it involves a forced choice (Copeland Reference Copeland2000; Moor Reference Moor2001).
In addition, the three-player game turns an interrogator’s natural disposition to anthropomorphize into a controlled variable, thereby preventing it from compromising the test. In his 1948 report, Turing remarked that interacting with even a “paper machine” produces “a definite feeling that one is pitting one’s wits against something alive” (1948, 412). This is another astute observation: humans frequently treat chatbots as animate, in particular those designed as “AI companions” with “personality” attributes (Østergaard Reference Østergaard2025). Several current researchers posit an “AI psychosis”, where users of agentic AI come to believe that these systems are conscious beings (Morrin et al. Reference Morrin, Nicholls, Levin, Yiend, Iyengar, DelGuidice, Bhattacharyya, MacCabe, Tognin, Twumasi, Alderson-Day and Pollak2025). The result of humans’ disposition to anthropomorphize is that no two-player imitation game can constitute a reliable test: the interrogator might simply be fantasizing. The three-player setup, however, is not vulnerable in the same way: we may presume that the interrogator’s inclination to anthropomorphize is present in interactions with the machine and with the human (Proudfoot Reference Proudfoot2011).
4.2 Mitigating the effect of irrelevant attributes
In Mind, Turing said that the computer can “deliberately introduce mistakes in a manner calculated to confuse the interrogator” (1950, 448). The machine is “permitted all sorts of tricks so as to appear more man-like, such as waiting a bit before giving the answer, or making spelling mistakes” (Turing et al. Reference Turing, Braithwaite, Jefferson and Newman1952, 495). This aspect of Turing’s design—allowing the programmer to hide the machine’s perfect spelling or fast response time—is often regarded as a defect: his test, critics claim, merely encourages “artificial stupidity” in the machine.Footnote 17 However, incorporating such “tricks” is necessary and proper.
In the late 1940s and early 1950s, newspapers commented, often with photographs, on the “somewhat unprepossessing” appearance of the Manchester machine (the successor to the Manchester “Baby”, the world’s first electronic stored-program digital computer).Footnote 18 This computer, they said, is “a mass of untidy wires, valves, chassis, and display tubes”.Footnote 19 The Cambridge EDSAC (the second electronic stored-program computer to function) was also described in unflattering terms—as a “vast number of valves set in grey-painted metal racks”, which could be taken for “a valve-maker’s store or part of a spares room at the B.B.C.”.Footnote 20 These depictions of the early computers made them seem more like primitive mechanical calculators and less like thinking machines.Footnote 21
It is in the context of such media coverageFootnote 22 that Turing remarked, “We do not wish to penalise the machine for its inability to shine in beauty competitions” (1950, 435). This inability, he said, is an “irrelevant” property—that is to say, a machine’s “unprepossessing” appearance does not defeat the claim that it can think (1950, 435). It follows that, when testing a machine for intelligence, its appearance, along with all its other irrelevant properties (such as perfect spelling), should be disguised—if, as Turing entreated, “fair play” is to be given to the machine (Reference Turing1947, 394).Footnote 23
4.3 Identifying real understanding
Turing described his imitation game as “something like a viva-voce examination” (1951, 484). Several recent theorists claim that a viva is “just one way” of structuring the game—and also that this format is undesirable, since it puts “too much emphasis” on the computer’s ability to “articulate one’s reasons for one’s actions” (Bohn Reference Bohn2024, 8; Allen et al. Reference Allen, Varner and Zinser2000, 254). Increasingly, experiments described as Turing tests simply provide an interrogator with (say) transcripts of text previously produced by computers and humans; the interrogator is to identify, based only on the transcripts, which authors are machines (e.g., Aharoni et al. Reference Aharoni, Fernandes, Brady, Alexander, Criner, Queen, Rando, Nahmias and Crespo2024; Dillion et al. Reference Dillion, Mondal, Tandon and Gray2025; Mei et al. Reference Mei, Xie, Yuan and Jackson2024).
Although these tests are logistically easier, losing the opportunity to challenge the machine is a major disadvantage (Proudfoot Reference Proudfoot2024). This is because, as Turing said, the viva format can be used “to discover whether some one really understands something or has ‘learnt it parrot fashion’”—that is, merely by rote memorization (1950, 446). It is well known that an LLM may perform well due only to “spurious correlations”—where, for example, a network with added image processing is being trained to identify images of boats but mistakenly takes a harbour in the background of several images as the salient feature (Hosseini et al. Reference Hosseini, Nawathe, Moayeri, Balasubramanian and Feizi2025). In this case, even though the model might perform well on some boat image tests, it does not “really understand” what a boat looks like and may go on to mislabel any image with a harbour background as an image of a boat. In Turing’s viva, the interrogator can require a machine to explain and justify its behaviour, thereby reducing the chances of its doing well due solely to spurious associations.
The importance of the viva format also has implications for the choice of interrogators—who, Turing said, “should not be expert” about machines (Turing et al. Reference Turing, Braithwaite, Jefferson and Newman1952, 495). What might Turing have meant by “expert”? Since he was speaking to the Third Programme’s diverse audience, we might presume that that he used “expert” as a vague term—and similarly that, when predicting in Mind how an average interrogator might do, he did not intend “average” as a precise statistical measure. Turing sought primarily to exclude interrogators with knowledge of a computer’s abilities and architecture, rather than to include a certain sort of interrogator.Footnote 24 This exclusion is usually ignored; for example, in recent online imitation games, typically interrogators were selected without controlling for expertise, and in the much-publicized 2014 Royal Society Turing Test (where the Eugene Goostman chatbot was declared to pass the test), several were computer scientists.
Indeed, Turing’s restriction may seem a fault in his test, since one might expect computer scientists to be the most perceptive interrogators. However, the situation is more complicated. In the Loebner Prize Contests, for example, knowledgeable interrogators probed specific weaknesses of AI systems by replacing ordinary conversation with gibberish, peculiar “common knowledge” questions, and syntactically tricky sentences quite unlike everyday language. These strategies deny a machine the opportunity to give, as Turing required, a “satisfactory and sustained” viva performance (1950, 447). Moreover, programmers reacted to these strategies by incorporating canned replies to such odd inputs, typically non sequiturs (e.g., Eugene Goostman, when unable to answer a question, frequently asked the interrogator for their address or occupation).Footnote 25 Expert interrogators can, then, hinder the practical role of Turing’s game in improving AI capabilities.
5. Experimental implementation and results: An informal description
For any thought experiment or actual implementation to be Turing’s test, we have argued, it should possess the design features established in sections 2–4. Yet one might ask: why should scientists pay attention to Turing’s test?Footnote 26 Benchmarks (e.g., Guo et al. Reference Guo, Xu, Zhang, Song, Peng, Deng, Dong, Nakayama, Geng, Wang, Ni, Yang, Rao, Peng, Hu, Wetzstein and Hu2025) that are currently used for “reasoning”, “common sense”, “natural language processing”, “logical thinking”, “theory of mind”, and suchlike in LLMs appear to provide criteria of intelligence that are much simpler, logistically and conceptually, than Turing’s test. In addition, neuroscientists (e.g., Kazemian et al. Reference Kazemian, Elmoznino and Bonner2025) offer the promise of building artificial neural networks with architectural similarities to human brains, and consequently of testing machines for those similarities. Turing’s test has an important role as one of the philosophical foundations of AI. Apart from this, however, what reason could a scientist have for running Turing’s test?Footnote 27
Turing’s philosophy of mind may provide a reason why his test is not redundant, even assuming vastly improved versions of current modes of testing generative AI models. (It seems unlikely that Turing simply did not envisage automated testing.) In his 1948 paper, Turing said that the concept of intelligence is an “emotional concept” and the idea of “intelligence” is “itself emotional rather than mathematical” (1948, 431, 411). On a recent interpretation of this and other remarks, the philosophical justification for his test is (what we would now call) a response-dependence approach to the mind: whether an entity is intelligent or thinks is in part determined by how human beings respond to the entity, including in the conditions of the imitation game (Proudfoot Reference Proudfoot2013).Footnote 28 Neither standard LLM benchmarking nor neuroscience-inspired AI involves responses in the way suggested by Turing’s remarks—so, neither could replace his test as a test for “intelligence” or “thinking” in the response-dependence sense.Footnote 29
The other well-known interpretations of Turing’s philosophy of mind—behaviourism and (classical or non-classical) computationalism—appear to take intelligence or cognition as response-independent. On those readings, improved automated testing or neuroscience might properly replace the imitation game. Ultimately, then, motivating the implementing of Turing’s test might require deciding fundamental questions in the philosophy of mind regarding the concept of intelligence or thinking.
For now, however, we may have good reason to run Turing’s test. Perhaps (as suggested in section 4.3) the imitation game can check that LLMs that do well on “reasoning” benchmarks are in fact capable of reasoning (see, e.g., Mondorf and Plank Reference Mondorf and Plank2024 on LLM failures). Or perhaps the game will prove an efficient means of testing for several cognitive abilities at once. Even the scientist who is sceptical of the test’s traditional role might regard Turing’s test as a useful stopgap until artificial networks become significantly more brain-like.
5.1 Testing an LLM
In our study, we carried out a computer-imitates-human game (CIHG), with GPT-4-Turbo as the computer. (As noted in the introduction, here we give only a brief informal report of our study; full technical details are available in Temtsin et al. Reference Temtsin, Proudfoot, Kaber and Bartneck2025.) Our CIHG had three players, and the interrogator could initiate conversation on any topic. The game was open-ended; the length of each trial was determined by the interrogator, who continued the interaction until confident of identifying the computer. We excluded interrogators and interviewees who might reasonably be expected to have some AI (or philosophy of AI) expertise.Footnote 30
In his computer-imitates-human game, Turing said, the machine can “brag” about its “charms, strength or heroism”—just as the (mid-twentieth century English) man in the man-imitates-woman game can say falsely that his hair “is shingled, and the longest strands are about nine inches long” (1950, 435, 433). Analogously, GPT-4-Turbo was prompted to behave as an English-speaking university student, using colloquial language and idioms, and to deny that it was a machine. It was also prompted to delay its responses.Footnote 31
In addition, we implemented Turing’s man-imitates-woman game (MIWG). Doing so enabled us to score the CIHG using each of the two frameworks (discussed in section 3) that are not ruled out by Turing’s own words. Our MIWG was another three-player game, on any conversational topic, with the duration determined in the same way by the interrogator. For consistency with the CIHG, we excluded participants from the MIWG who could reasonably be expected to have knowledge of gender differences in interaction or communication style.
All other aspects of the study design conformed to guidelines for good practice.
To the best of our knowledge, no previous empirical study conforms to Turing’s conceptual design. For example, although the latest study claiming success in the Turing test (a Prolific replication study reported in Jones and Bergen Reference Jones and Bergen2026) implemented a three-player imitation game and increased the time limit to 15 minutes, it did not exclude all participants who might be presumed to be aware of AI debates or the test,Footnote 32 and it retained a set duration for the game. In addition, this study did not implement a man-imitates-woman game (in the study, a machine passed the test if interrogators did not do significantly better than chance).
5.2 The duration issue
In our study, 33 out of 37 interrogators—89%—in the CIHG took longer than five minutes to make a decision. The average (mean) duration of CIHG trials was 14 minutes, and the longest trial lasted 46 minutes. Typically the longer a task takes, the greater the cognitive resources required for that task (Barrouillet et al. Reference Barrouillet, Bernardin, Portrat, Vergauwe and Camos2007).Footnote 33 It would appear, then, that most CIHG interrogators need, or believe that they need, considerably longer than five minutes in order to come to a decision.Footnote 34
This result supports individual interrogators’ critical comments regarding earlier “Turing tests”. For example, Barry Cooper said of the 2014 Royal Society Turing test that he felt “confusion skewed the outcome”, remarking that “of course, the 5 minutes was for 2 conversations, so one had little more than 2 minutes with the machine” (Cooper, personal communication with Proudfoot, 2014). Similarly, Dan Falk said about the 2012 Bletchley Park Turing Test: “five minutes goes by pretty fast. And the shorter the conversation, the greater the computer’s advantage” (Falk Reference Falk2014).Footnote 35
Even setting aside Turing’s intention, then, our study suggests that the practice of setting a five-minute (and even a 15-minute) limit to his imitation game may hamper interrogators, potentially invalidating the results of any game.
5.3 The machine’s performance (poor)
In the CIHG, 36 out of 37 interrogators—97%—correctly identified the computer. So, using Moor’s scoring framework (namely, that the average interrogator should not do significantly better than chance), GPT-4-Turbo did not pass the test.Footnote 36 In addition, 16 out of 37 interrogators in the MIWG—43%—correctly identified the man. Consequently, the performance of the average CIHG interrogator was very much better than the performance of the average MIWG interrogator; so, using Copeland’s scoring framework (namely, that the CIHG interrogator not do better than the MIWG interrogator), again GPT-4-Turbo did not pass the test.
Of course, like some recent Turing test studies (Rahimov et al. Reference Rahimov, Zamler and Azaria2025; Restrepo Echavarría Reference Restrepo Echavarría2025), we tested only one model. Also like some studies (Jones and Bergen Reference Jones and Bergen2024; Rahimov et al. Reference Rahimov, Zamler and Azaria2025;Footnote 37 Restrepo Echavarría Reference Restrepo Echavarría2025), we used only one prompt (per model). Different models or prompts might produce a better result for the machine. New generative AI models are released frequently (at the date of writing, the latest available GPT model is the GPT-5.6 family). What might the results of our study have been, had we used GPT-5.6 Sol, or the most recent Claude or Gemini iteration—let alone an LLM under development now (such as Open AI’s Astra) or in the future? It is impossible to say: the literature on improvements in LLMs is complicated, and it appears difficult systematically to compare the performance of different models (Afridi et al. Reference Afridi, Ullah, Khan and Ullah2026). How well either a different LLM already heralded as a Turing-test passer or a model later than GPT-4-Turbo would do in our CIHG is an entirely open question, to be answered experimentally.
GPT-4-Turbo’s poor performance does not indicate any inherent difficulty in an LLM’s passing Turing’s test. Several recent discussions claim that no LLM genuinely cognizes—cognition is said to require, for example, consciousness (e.g., Hadid et al. Reference Hadid, Jerbi and Krakauer2026), sensory grounding or embodiment (e.g., Bender and Koller Reference Bender, Koller, Jurafsky, Chai, Schluter and Tetreault2020; Gahrn-Andersen Reference Gahrn-Andersen2025), or a hybrid architecture incorporating symbolic AI (e.g., Marcus and Davis Reference Marcus and Davis2019). We take no position here with respect to these larger debates. Rather, we provide experimenters with a modest challenge: to earn the cachet of passing Turing’s test, their LLM must do well in the CIHG.
5.4 The gender test revisited
The average (mean) duration of the MIWG trials was 24 minutes—ten minutes longer than the average for CIHG interrogators. The longest trial lasted 49 minutes.
Several scholars claim that, irrespective of Turing’s intention, a computer-imitates-woman test is “vastly superior” to a computer-imitates-human test (Sterrett Reference Sterrett2000, 542; see also Sterrett Reference Sterrett2020). The former, it is said, is “more stringent” than the latter, which is “a weaker game for the computer to play” (Traiger Reference Traiger2000, 570). A reason given for the superiority of the gender test is that female impersonation is (it is claimed) especially challenging for the impersonator, whether a man or a machine: it demands “more of what is relevant to thinking”, including “resourcefulness in dealing with unfamiliar tasks” and “a critical attitude to one’s habitual responses” (Sterrett Reference Sterrett2000, 547, 542, 541).
Our study offers no empirical support for these theoretical claims. As noted above, longer response times typically indicate a more demanding task. So, with respect to the demand our female impersonation test (the MIWG) placed upon a player’s cognitive resources, our study points to a demand only on the interrogator (as compared with the CIHG interrogator), and not on the impersonator. In addition, if a female impersonation test were especially demanding, then, all things being equal, we might expect more MIWG interrogators than CIHG interrogators to make correct identifications. In our study, in contrast, CIHG interrogators were markedly more successful.
6. Conclusion
Numerous accounts profess to analyse or implement Turing’s test. In this paper we aimed to put to bed some very well-trodden interpretative debates and to provide a conceptual blueprint for future implementations. Only a test that instantiates the design identified here, we have argued, can claim to be Turing’s test.
A fortiori, only an unintelligent (unthinking) machine that passes such a test provides a counterexample to Turing’s criterion for thinking. In our view, no counterexample to the test has been demonstrated. Of the numerous proffered counterexamples, many are imaginary machines; for example, Ned Block famously claimed that his hypothetical gigantic lookup table machine, possessing the intelligence merely of a “toaster”, could pass the test (Reference Block1981, 21). Turing made clear, however, that his proposal concerned real-world machines performing in real time—with the result that merely logically possible machines are not counterexamples (Turing et al. Reference Turing, Braithwaite, Jefferson and Newman1952, 503; Copeland Reference Copeland2000; Proudfoot Reference Proudfoot2013). As to actual machines, recent generative AI models have been said finally to “sail through” the test (Gibney Reference Gibney2025);Footnote 38 and many theorists state that these machines “do not think”—they have “great memories” but produce only “inane fabrications” (Johnson-Laird and Ragni Reference Johnson-Laird and Ragni2023; Berghel Reference Berghel2025, 141). However, to the best of our knowledge, no LLM study conforms to Turing’s design, and so none provides a counterexample. In sum, studies to date provide no empirical evidence that a new test of intelligence in machines is, as is increasingly often claimed, “urgently required” (Chaudhri Reference Chaudhri2025).
As to our own study, the results of implementing for the first time Turing’s test as he proposed it are consistent with his 1952 prediction that it would be at least a century before a machine passed the test. Nevertheless, our study implemented Turing’s conceptual blueprint using a specific LLM, prompt, and participants. Like all empirical studies, this design must be repeated with different models, different prompts, different subjects, and other relevant variations.
Acknowledgements
We would like to thank three anonymous Philosophy of Science reviewers, whose suggestions substantially improved the paper.
Funding statement
None to declare.
Declaration
None to declare.
Open access
