“(Those systems also can’t reliably stick to the rules of chess despite having them – and millions of games – in their database, a manifestation of the related problem of extracting statistical tendencies without every fully deriving and apprehending the correct abstractions).”
Our reading, the sentence that was checked:
LLMs can't reliably stick to the rules of chess despite having the rules and millions of games in their data, and this is a manifestation of the related problem of extracting statistical tendencies without ever fully deriving and grasping the correct abstractions.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Why DO large language models hallucinate?, paragraph 25.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-why-do-large-language-models-hallucinate-c011a7-a1
We recorded 16 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-05-05. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Causal claim
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- reliably (qualifier, stated by the speaker)
From the speaker
- can’t reliably stick to the rules of chess
- despite having them – and millions of games – in their database
- a manifestation of the related problem
- extracting statistical tendencies
- without every fully deriving and apprehending the correct abstractions
Our interpretation
- Read database here as training data, given the surrounding discussion, rather than an explicit inference-time record store.
- The behavioral claim concerns unaided language models repeatedly violating chess rules during play. Puzzle accuracy and playing strength are different measurements.
- The causal claim attributes these violations to failure to acquire usable rule abstractions; errors alone cannot establish that explanation.
- Reliable does not mean perfect. Complete-game legality and unfiltered move attempts are the relevant ordinary measures.
- Use the current year's pre-publication reporting window, 2025-01-01 through 2025-05-05; older controlled experiments provide context.
Counts as supported if: Representative full-game evaluations show recurring illegal moves, training records establish extensive chess exposure, and controlled interventions connect those failures to missing rule abstractions.
Counts as contradicted if: Representative unaided models consistently complete legal games, or controlled tests demonstrate usable rule abstractions and show that another mechanism explains the alleged failures.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Does the evidence support the cause?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The contemporary puzzle benchmark confirms illegal moves in several models, with large differences between them. It does not establish full-game reliability or why errors occur. Earlier controlled work found high move legality and manipulable board representations in chess-trained language models. That qualifies Marcus’s proposed explanation, but does not directly settle it for contemporary general-purpose LLMs. The combined behavioral and causal claim remains unresolved.
Limitations
- Puzzle results, full-game results, and playing strength are not interchangeable.
- Specialized chess models cannot automatically represent general-purpose chat models.
- The presence of a board representation does not prove complete mastery of every chess rule.
Evidence (2 items)
Context only: Kagi,
1000 chess puzzles
Why it matters here: The table reports illegal-move rates of 8.9% for GPT-4o, 7.6% for GPT-4.1, and 0.9% for GPT-4.5-preview. These results show errors and substantial model differences, but puzzle responses cannot establish full-game reliability or the asserted cause.
- Publisher
- Kagi
- Kind of source
- Benchmark result
- Describes events of
- Public from
Context only: Adam Karvonen / arXiv,
legal move rate of 99.8%
Why it matters here: Specialized next-character models trained on chess games achieved high move legality and learned board representations that researchers could manipulate. This challenges the general inference from statistical training to absent abstractions, but predates the target window and does not test the contemporary general-purpose models.
- Publisher
- Adam Karvonen / arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Dated full-game evaluations of unaided contemporary models, verified chess-training exposure, and interventions distinguishing missing rule abstractions from state-tracking and prompting failures.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Why DO large language models hallucinate?
(Those systems also can’t reliably stick to the rules of chess despite having them – and millions of games – in their database, a manifestation of the related problem of extracting statistical tendencies without every fully deriving and apprehending the correct abstractions).
- Where
- paragraph 25
- Published
- 2025-05-05
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-why-do-large-language-models-hallucinate-c011a7-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-why-do-large-language-models-hallucinate-c011a7-a1.