“They do not gravitate towards human language, and they never develop a cohesive undestanding of the world, no matter how much experience they have.”
Our reading, the sentence that was checked:
Systems like GPT-3 do not gravitate towards human language, and they never develop a cohesive understanding of the world, no matter how much experience they have.
Assessment: Contradicted Settled before its deadline on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Noam Chomsky and GPT-3, paragraph 23.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-noam-chomsky-and-gpt-3-61b596-a1
We recorded 8 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-05-21. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- Settled before its deadline
- Deadline state on 2026-10-29
- Settled before its deadline (the release’s reference date)
Conditions
- systems like GPT-3 (scope, stated by the speaker)
- no matter how much experience they have (qualifier, stated by the speaker)
From the speaker
- Systems like GPT-3
- do not gravitate towards human language
- never develop a cohesive undestanding of the world
- no matter how much experience they have
Our interpretation
- I read this as two categorical claims about a model family, including a permanent limitation, without a deadline.
- I interpret systems like GPT-3 to include GPT-style autoregressive transformer language models, including experimentally trained GPT-2 models. This does not restrict the claim to the released GPT-3 checkpoint.
- The ordinary reading of gravitate towards human language is a learning preference for human-language patterns over appropriate artificial-language controls. It does not require an exclusively human learning capacity.
- Cohesive understanding means reliable, generalizable tracking of objects, events and their spatial, temporal and causal relationships. Success in one restricted environment is insufficient.
- A demonstrated counterexample to either categorical component contradicts the combined assertion. It does not establish that the other component is false.
- No matter how much experience is a universal qualification, not a conditional antecedent requiring a particular training run.
Counts as supported if: Evidence establishing both limitations throughout the stated model family and across additional training experience. Finite examples of failure can support present limitations but cannot establish never.
Counts as contradicted if: Replicated controlled evaluations demonstrate a learning preference for human-language patterns in the stated model family, or demonstrate broad, consistent and generalizable world understanding without supplying a separate symbolic world model.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Contradicted
The dated evidence goes against what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- Settled before its deadline
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The language-learning component has a controlled counterexample and independent corroboration. GPT-2 experiments show some preference for human-language patterns over altered alternatives. That contradicts the categorical absence of such a tendency under the stated reading of systems like GPT-3. The studies do not show that models learn exactly like humans. Hunter's criticism limits the stronger grammatical interpretation. The separate claim about never developing cohesive world understanding remains unsettled: finite failures cannot prove never, and restricted simulation successes do not establish broad understanding.
Limitations
- The contradiction applies to the language-learning component of the combined assertion.
- Including GPT-2 experiments within systems like GPT-3 is an explicit scope interpretation.
- Preference for human-language patterns is weaker than a complete human-like language faculty.
- Experimental dates are reported at year precision; availability dates identify the cited versions.
Evidence (5 items)
Goes against the claim: arXiv,
struggles to learn
Why it matters here: Kallini and colleagues compared GPT-2 learning across English and systematically altered languages, including different training stages. The observed preference conflicts with the language-learning component under the stated family-level interpretation.
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Goes against the claim: arXiv,
human-like inductive biases
Why it matters here: Yang and colleagues extended testing to 12 languages from four families and found partial discrimination between attested and impossible patterns. This independently corroborates a preference, while explicitly rejecting equivalence with human learning.
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: arXiv,
does not actually test
Why it matters here: Hunter identifies a confound between counting rules and non-adjacent dependencies. This limits claims that the initial experiment demonstrates specifically human grammatical constraints, although that stronger conclusion is not required by the ordinary preference criterion.
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: arXiv,
incoherence creates fragility
Why it matters here: Vafa and colleagues find weaknesses in world-model coherence across game, logic and navigation settings. These findings support concern about tested systems but cannot establish failure at every future training scale.
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: arXiv,
implicit text-based world models.
Why it matters here: Li and colleagues report coherent state tracking and useful simulation in five text environments after targeted training. The restricted environments and dependence on training coverage prevent treating this as decisive evidence of broad world understanding.
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Broad controlled evaluations of state tracking, causal reasoning and transfer across unfamiliar domains would address the remaining world-understanding component.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Noam Chomsky and GPT-3
They do not gravitate towards human language, and they never develop a cohesive undestanding of the world, no matter how much experience they have.
- Where
- paragraph 23
- Published
- 2022-05-21
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-noam-chomsky-and-gpt-3-61b596-a1: Current. Outcome: Contradicted. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.