“ChatGPT gets them wrong because it is munging words together, not genuinely processing an (elementary) model of British PM history.”
Our reading, the sentence that was checked:
ChatGPT gets the British prime minister tasks wrong because it is munging words together rather than genuinely processing an elementary model of British PM history.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, LLMs are not like you and me—and never will be., paragraph 20.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-llms-are-not-like-you-and-meand-never-3c190c-a1
We recorded 11 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-08-12. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Causal claim
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- the British prime minister list tasks discussed (scope, stated by the speaker)
From the speaker
- ChatGPT gets them wrong because
- munging words together
- not genuinely processing an (elementary) model of British PM history
Our interpretation
- Them refers to the reported British prime minister name-filtering task, including omissions and an allegedly invalid inclusion.
- The plain reading is that failure to use a usable representation of the relevant historical facts caused the errors.
- The claim concerns the reported GPT-5 behavior in August 2025, not whether every language model can represent any historical information.
- Finding errors is insufficient to distinguish missing knowledge from retrieval, name selection, letter filtering, instruction following, or answer-generation failures.
Counts as supported if: Reproducible task-specific experiments and mechanistic interventions show that the errors arise from absent or unused historical representations, rather than competing causes.
Counts as contradicted if: Task-specific evidence establishes that the model used the relevant historical representation and that another mechanism caused the reported errors.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Does the evidence support the cause?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The reported errors would establish unsuccessful output if authenticated. They would not establish why it failed. Research on other models shows that structured representations can exist, but cannot determine what GPT-5 used in this task. The specific causal explanation therefore remains unverified.
Limitations
- Evidence from Llama-2 cannot directly support or contradict a claim about GPT-5's internal processing.
- Behavioral success or failure alone does not uniquely identify an internal mechanism.
Evidence (1 item)
Context only: arXiv; Wes Gurnee and Max Tegmark,
linear representations of space and time
Why it matters here: The study finds structured representations in Llama-2 models, cautioning against inferring the absence of representations from text generation alone. It does not establish what GPT-5 represented or used in the British prime minister task.
- Publisher
- arXiv; Wes Gurnee and Max Tegmark
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
The original task record and reproducible interventions that separate historical representation from retrieval, letter filtering, and exhaustive listing.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, LLMs are not like you and me—and never will be.
ChatGPT gets them wrong because it is munging words together, not genuinely processing an (elementary) model of British PM history.
- Where
- paragraph 20
- Published
- 2025-08-12
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-llms-are-not-like-you-and-meand-never-3c190c-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-llms-are-not-like-you-and-meand-never-3c190c-a1.