Caricaturing Noam Chomsky
We recorded 5 claims from this post. 4 were checked against evidence. 1 can’t be checked. 6 more are held. Read the post at its source.
Claims from this post (5)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
“Still, even though Chomsky’s argument surely could have used considerably more nuance (see my article with Ernest Davis on how not to evaluate GPT), his overall point is correct: LLMs don’t reliably understand the world.”
What we tested: LLMs do not reliably understand the world.
What happened: The developer of the leading LLM of early 2023 says the model, like earlier GPT models, is not fully reliable: it hallucinates facts and makes reasoning errors. This matches the behavioural reading of 'don't reliably understand the world'. (OpenAI (arXiv), )
Assessment: UnresolvedSaid
“Minerva, for example, is trained on an immense amount of data, and ostensibly captures two-digit arithmetic, but it fails altogether on 4-digit multiplication,”
What we tested: Minerva, despite immense training data and apparent two-digit arithmetic ability, fails altogether on 4-digit multiplication.
Assessment: UnresolvedSaid
“The same goes for essentially everything an LLMs talks about; there is a degree of generalization to similar examples, but distribution shift is hard (the crux of my own work going back to 1998), and nearly any generalization can be easily broken.”
What we tested: For essentially everything an LLM talks about, there is a degree of generalization to similar examples, but distribution shift is hard and nearly any generalization can be easily broken.
Assessment: UnresolvedSaid
“The request is perfectly clear, not a nonsensical question in any way. The prompt is perfectly sensible; the system just isn’t up to the job.”
What we tested: The request in his example is perfectly clear and sensible, not nonsensical in any way, and the system just isn't up to the job.
Assessment: Can’t be checkedSaid
“But there is no real argument against Chomsky’s overall take: these systems are useful technology but remain a long way from true artificial intelligence, and they are even further from telling us anything useful about the function of the human mind.”
What we tested: LLMs are useful technology but remain a long way from true artificial intelligence and even further from telling us anything useful about the human mind.
Held from this post (6)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: What Marcus said and what Chomsky said is correct: the system (ChatGPT) really isn't up to the job.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: LLMs certainly have not taught us anything whatsoever about why the world is as it is rather than some other way, and the same goes for why the human mind is as it is.
- The two reviews reached different assessments. Our reading, not assessed: ChatGPT is wildly stochastic and unreliable, so single examples of its behavior prove little.
- The criteria are ambiguous. Our reading, not assessed: Minerva's arithmetic failures are most parsimoniously explained by pattern recognition over stored examples rather than genuine understanding of multiplication.