“(They also can’t play chess as well as conventional algorithms, can’t fold proteins like special-purpose neurosymbolic hybrids, can’t run databases as well as conventional databases, etc.)”
Our reading, the sentence that was checked:
LLMs can't play chess, fold proteins, or run databases as well as conventional or special-purpose systems.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, A knockout blow for LLMs?, paragraph 33.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-a-knockout-blow-for-llms-753c69-a3
We recorded 19 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-06-07. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- can’t play chess as well as conventional algorithms
- can’t fold proteins like special-purpose neurosymbolic hybrids
- can’t run databases as well as conventional databases
Our interpretation
- The ordinary reading compares general-purpose LLMs performing these tasks themselves with competent specialist systems. Calling a chess engine or database does not establish the LLM's own performance.
- Assess chess playing strength, protein-structure prediction accuracy, and database-operation correctness and performance separately. Supporting the entire statement requires evidence for all three.
- The present-tense statement describes capabilities available before the post. With no standard capability-reporting period, use calendar 2025 through June 7 as the contemporary observation window; older results are context.
- The claim does not establish that the specialist protein systems are correctly classified as neurosymbolic.
Counts as supported if: Comparable evaluations of contemporary general-purpose LLMs show materially inferior performance to competent specialist systems in all three named domains.
Counts as contradicted if: Comparable evaluations establish that contemporary general-purpose LLMs match or exceed competent specialist systems in one or more named domains. Report which component is contradicted; do not infer the other components.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The January 2025 chess study reports an Elo rating of 1788 for a specially trained ChessLLM, evaluated against reduced-strength Stockfish settings [e1]. It does not establish how contemporary general-purpose LLMs compare with full-strength chess engines, and it cannot settle the three-domain claim [e1].
Limitations
- The packet supplies no qualifying protein-structure or database-execution comparison.
- The chess evidence establishes neither parity nor inferiority for contemporary general-purpose LLMs against competent specialist engines.
- Missing comparisons do not establish that the claim is false.
Evidence (1 item)
Context only: Zhang et al., arXiv,
Elo rating of 1788
Why it matters here: The paper reports a fine-tuned chess model competing against reduced-strength Stockfish settings. It establishes meaningful chess capability, but neither parity with full-strength engines nor the performance of contemporary general-purpose LLMs.
- Publisher
- Zhang et al., arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Reproducible comparisons applicable to January 1–June 7, 2025, specifying model versions, specialist baselines, resources, and tool access. Tests should measure full-game chess strength, experimentally grounded protein-structure accuracy, and database execution correctness and performance. Inferiority in all three domains would support the whole claim; parity or superiority in any domain would contradict that component.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, A knockout blow for LLMs?
(They also can’t play chess as well as conventional algorithms, can’t fold proteins like special-purpose neurosymbolic hybrids, can’t run databases as well as conventional databases, etc.)
- Where
- paragraph 33
- Published
- 2025-06-07
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-a-knockout-blow-for-llms-753c69-a3: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.