“Large language models are (unless hybridized with symbolic systems, a la Jurassic X) are, on their own, inherently weak at understanding the physical and psychological world.”
Our reading, the sentence that was checked:
LLMs on their own, without symbolic hybridization, are inherently weak at understanding the physical and psychological world.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, What does it mean when an AI fails? A Reply to SlateStarCodex’s riff on Gary Marcus, paragraph 51.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-what-does-it-mean-when-an-ai-fails-a78bb4-a1
We recorded 14 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-06-07. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- unless hybridized with symbolic systems (qualifier, stated by the speaker)
From the speaker
- Large language models
- unless hybridized with symbolic systems
- on their own
- inherently weak
- understanding the physical and psychological world
Our interpretation
- The ordinary behavioral reading is a systematic weakness in reasoning about physical situations and people's beliefs, intentions and reactions, rather than a claim that models answer every such question incorrectly.
- Inherently attributes the weakness to the unaided model approach; isolated mistakes are insufficient to establish that stronger component.
- Symbolic solvers and external reasoning systems are excluded. Natural-language prompting alone does not count as symbolic hybridization.
- The observation concerns models available by the post. Its contextual reference to continuing general phenomena supplies the August 2020-to-June 2022 historical interval.
- Later models cannot directly decide this historical observation.
Counts as supported if: Varied evaluations of contemporary unaided LLMs show systematic weaknesses in both physical and psychological reasoning, with evidence that those weaknesses belong to the approach rather than a particular benchmark or configuration.
Counts as contradicted if: Contemporary unaided LLMs demonstrate robust, generalizable competence across varied physical and psychological reasoning tasks, including novel cases and controlled variations.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The evidence gives reasons for concern about physical reasoning and reasons to recognize substantial commonsense competence. Neither establishes the full claim about inherent weakness across both physical and psychological understanding in contemporary unaided LLMs. Calling it supported would generalize beyond the tested models and domains; calling it contradicted would treat benchmark success as broad robustness.
Limitations
- TRIP tests a narrower model population than the frontier generative systems discussed in the post.
- PaLM's favorable results do not by themselves demonstrate robust understanding.
- The word inherently makes the claim stronger than an observation of particular errors.
Evidence (2 items)
Context only: Shane Storks, Qiaozi Gao, Yichi Zhang and Joyce Chai / arXiv,
supporting evidence.
Why it matters here: TRIP documents difficulties connecting physical story judgments to supporting states and conflicts. Its BERT-family systems and physical-only coverage do not establish the complete claim about contemporary leading LLMs and both domains.
- Publisher
- Shane Storks, Qiaozi Gao, Yichi Zhang and Joyce Chai / arXiv
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
Context only: Google researchers / arXiv,
reasoning tasks.
Why it matters here: PaLM's commonsense results show meaningful competence using natural-language prompting, limiting a blanket reading of weakness. The combined statement also covers calculator-assisted arithmetic, and does not establish robust competence across both physical and psychological reasoning.
- Publisher
- Google researchers / arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Broad, controlled evaluations of the contemporaneous models across both domains, with novel task variations and documented absence of symbolic assistance, together with evidence distinguishing systematic model limitations from benchmark-specific failures.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, What does it mean when an AI fails? A Reply to SlateStarCodex’s riff on Gary Marcus
Large language models are (unless hybridized with symbolic systems, a la Jurassic X) are, on their own, inherently weak at understanding the physical and psychological world.
- Where
- paragraph 51
- Published
- 2022-06-07
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-what-does-it-mean-when-an-ai-fails-a78bb4-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-what-does-it-mean-when-an-ai-fails-a78bb4-a1.