Nope, the Turing Test has not been solved
We recorded 4 claims from this post. 4 were checked against evidence. Read the post at its source.
Claims from this post (4)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
“ELIZA (built in 1965-1966) beat GPT 3.5.”
What we tested: In the cited Turing Test study, the 1960s ELIZA chatbot outperformed GPT-3.5.
What happened: The same results paragraph reports ELIZA at 27%, above both GPT-3.5 configurations. These are overall judged-human rates in the experiment Marcus cited. (arXiv; Cameron Jones and Benjamin Bergen, )
Assessment: SupportedSaid
“And, sorry Sam, when you actually look at the data. humans are still ahead.”
What we tested: When you actually look at the data in the cited Turing Test study, humans are still ahead of the AI systems tested.
What happened: The section reports a 63% human success rate and 41% for the strongest GPT-4 prompt. All other tested AI configurations scored lower. (arXiv; Cameron Jones and Benjamin Bergen, )
Assessment: UnresolvedSaid
“What I proposed then, as a replacement to the Turing Test, was a Comprehension Challenge; even now, no software would be able to meet that challenge.”
What we tested: As of December 2023, no AI software could meet Marcus's Comprehension Challenge.
Assessment: UnresolvedSaid
“Dubious Google Gemini videos aside, nobody is close to passing that yet.”
What we tested: As of December 2023, no one, Google's Gemini included, was close to passing the Comprehension Challenge.