How Not to Test GPT-3
We recorded 12 claims from this post. 7 were checked against evidence. 5 can’t be checked. 9 more are held. Read the post at its source.
Claims from this post (12)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
Observation
“The truth, though, is that GPT often fails in problems involving theory of mind.”
Our reading: GPT often fails at problems involving theory of mind.
No horizon Evidence as of
Limitation: The result depends on reading 'often' as recurring failures rather than a specified population failure rate.
Assessment: Can’t be checkedSaid
Causal claim
“Kosinski is probably tapping into stuff that is very heavily attested in the database.”
Our reading: GPT-3's success on Kosinski's theory-of-mind tests probably comes from the test material being heavily represented in its training data.
No horizon
Assessment: UnresolvedSaid
Forecast
“But In practice, that isn’t going to happen.”
Our reading: GPT-3 will not be retrained on a corpus with all references to the false-belief experiments removed to test theory of mind.
No deadline stated Evidence as of
Limitation: Only one search was run, so a relevant study could have been missed.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But in testing AIs, it is a really bad idea.”
Our reading: Reusing standard published examples from cognitive psychology is a really bad idea when testing AIs.
No horizon
Assessment: UnresolvedSaid
Observation
“However, just in terms of percentage right vs. wrong, the improvement was slight; It now got 50% right, 40% wrong, and 10% borderline (marginally better than the 45% right before).”
Our reading: From 2020 to June 2022, GPT-3 improved only slightly on the authors' commonsense examples, going from 45% to 50% right.
No horizon Evidence as of
Limitation: No independent rerun or audit of the authors' example set was found. That does not prove none exists.
Assessment: UnresolvedSaid
Observation
“For one thing, what we publish often gets incorporated into the training; perhaps in company benchmarks as well.”
Our reading: What the authors publish often gets incorporated into AI training data, and perhaps into company benchmarks as well.
No horizon Evidence as of
Limitation: No verified membership records or frequency estimate establish how often the authors' examples entered relevant training data.
Assessment: Can’t be checkedSaid
Observation
“Testing on items that have been directly published in the literature and then likely ingested doesn’t really tell us much.”
Our reading: Testing LLMs on previously published items that were likely ingested in training reveals little.
No horizon
Qualifier: items published and likely ingested
Assessment: UnresolvedSaid
Observation
“The problems in physical and psychological reasoning that we pointed in the Fall of 2020 very much remain.”
Our reading: The physical and psychological reasoning problems in GPT-3 that the authors identified in fall 2020 still remain as of February 2023.
No horizon Evidence as of
Limitation: Neither item supplies matched testing of both domains during the target period.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“DO NOT use examples or brain teasers that are very well known in the scientific or popular literature. These will certainly be in the AI’s training material, and it may have memorized the answer.”
Our reading: Testers should not use well-known examples or brain teasers to test AI, because these are in training data and may have been memorized.
No horizon
Assessment: SupportedSaid
Observation
“People had been testing these Ais abilities at simple arithmetic for years before it occurred to anyone to check whether they can count. (They can’t, reliably).”
Our reading: LLMs cannot count reliably.
No horizon Evidence as of
Limitation: Reliability has no speaker-specified threshold; a human-comparative interpretation would give more weight to Codex's strong result.
Assessment: SupportedSaid
Observation
“AI and AI test sets have a huge English-language bias.”
Our reading: AI systems and AI test sets have a huge English-language bias.
No horizon Evidence as of
Limitation: The assessment concerns language systems in the post's context, not all AI applications.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“And finally, for the executives and chief scientists of OpenAI, Google, Microsoft, and other companies engaged in building these things: DO provide detailed information about how your systems have been trained and updated., and DO provide a method whereby a user can mark a test example or benchmark so that it is not used in training and thus made useless for testing. To fail to do that is both socially and scientifically irresponsible.”
Our reading: OpenAI, Google, Microsoft and other AI developers should disclose how their systems are trained and updated, and should let users mark test examples to exclude them from training; not doing so is irresponsible.
No horizon
Held from this post (9)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The record does not meet the current rules. Our reading, not assessed: Apparently fixed LLM failures often reemerge with slight rephrasings or slightly different examples.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: Kosinski's study does not prove that GPT-3 has abstracted a generalized theory of mind, because its tests are well represented in training data.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: GPT-3's training data almost certainly included repeated descriptions of the classic false-belief experiments.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: LLMs have many critical weaknesses in the basic domains of time, space, and human interactions.
- The criteria are ambiguous. Our reading, not assessed: Large language models and other machine-learning-based AIs, such as machine translation systems, improve overall over time (since unimproved versions aren't released), but they don't improve across the board.