“Critically, as I argued at the end of June (and going back to 2019) LLMs never induce proper world models, which is why, for example, they still can’t even play chess reliably,1 and continue to make stupid, head-scratching errors with startling regularity.”
Our reading, the sentence that was checked:
LLMs never induce proper world models, which causes them to be unable to play chess reliably and to make frequent stupid errors.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, LLMs are not like you and me—and never will be., paragraph 4.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-llms-are-not-like-you-and-meand-never-14c423-a1
We recorded 10 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-08-12. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Causal claim
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- LLMs never induce proper world models
- which is why
- can’t even play chess reliably
- errors with startling regularity
Our interpretation
- Assess the explanation of existing capabilities at the post date, rather than treating 'never' as a separate forecast.
- A functional world model represents relevant states and relationships and uses them to guide predictions or actions.
- The causal assertion requires evidence connecting deficiencies in those representations to the specified errors. Observing errors alone is insufficient.
- The period is the accumulated capability record through 2025-08-12. No lower bound is supplied because the assertion concerns what LLMs have ever learned.
Counts as supported if: Controlled studies show that the relevant LLMs lack functional world representations and that this deficiency causes their chess failures and recurring errors, with plausible alternative explanations addressed.
Counts as contradicted if: Studies establish functional world representations in systems within the claim's scope and directly undermine the asserted explanation of their failures, or isolate a different explanation incompatible with the claimed causal mechanism.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Does the evidence support the cause?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The chess study supplies meaningful counterevidence to the claim that sequence models never learn functional representations. It does not settle the proposed explanation of general-purpose LLM failures. Neither failures alone nor a successful specialist model establishes the broad causal relationship Marcus asserts.
Limitations
- The strongest intervention evidence concerns a specialist chess model.
- The distinction between a bounded functional representation and a 'proper' world model affects the assessment.
Evidence (1 item)
Context only: arXiv; Adam Karvonen,
edit its internal board state.
Why it matters here: The study reports learned chess-state representations that respond to interventions, challenging a blanket absence premise. Its specialist next-character model does not establish the cause of errors across general-purpose LLMs; the event date records the demonstrated capability as publicly documented.
- Where
- Abstract
- Publisher
- arXiv; Adam Karvonen
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Controlled experiments on the relevant pre-post LLMs that measure their representations and selectively alter them to test whether those changes explain chess failures and other specified errors.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, LLMs are not like you and me—and never will be.
Critically, as I argued at the end of June (and going back to 2019) LLMs never induce proper world models, which is why, for example, they still can’t even play chess reliably,1 and continue to make stupid, head-scratching errors with startling regularity.
- Where
- paragraph 4
- Published
- 2025-08-12
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-llms-are-not-like-you-and-meand-never-14c423-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-llms-are-not-like-you-and-meand-never-14c423-a1.