“That’s exactly what it means to hit a wall, and exactly the particular set of obstacles I described in my most notorious (and prescient) paper, in 2022. Real progress on some dimensions, but stuck in place on others.”
Our reading, the sentence that was checked:
LLM development has hit a wall: real progress on some dimensions but stuck on others.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it., paragraph 84.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-gpt-5-overdue-overhyped-and-underwhelming-19e767-a1
We recorded 30 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-08-09. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- hit a wall
- obstacles I described
- in 2022
- Real progress on some dimensions
- stuck in place on others
Our interpretation
- Marcus explains his metaphor as quantitative progress alongside persistent qualitative failure categories. The ordinary contextual reading is not that benchmark scores or error rates stopped improving.
- The assessment concerns the empirical proposition about progress and continuing limitations. The rhetorical claim of prescience is not treated as independent evidence.
- The historical comparison runs from the referenced 2022 period through the post date. January 1 represents the beginning of that calendar year, not a claimed experimental date.
Counts as supported if: Comparative evaluations show improvements in some capabilities while newer models retain reasoning, factuality, or generalization failures of the kinds identified in the surrounding passage.
Counts as contradicted if: Representative evaluations show that newer models had eliminated the relevant qualitative failure categories, or that the asserted progress on other dimensions had not occurred.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The contextual proposition is supported: comparative evaluations show improvements while factuality and reasoning failures remain. OpenAI reports reduced hallucination rates, and independent ARC testing identifies continuing failures to apply rules in context. This supports Marcus's explicitly qualified account of progress alongside persistent weaknesses. It does not establish a quantitative plateau or prove that further scaling cannot overcome them.
Limitations
- The outcome depends on reading stuck as persistence of qualitative failures, as the surrounding passage indicates.
- The GPT-5 comparison is company-reported.
- Support does not extend to a permanent scaling barrier or every example in the surrounding passage.
Evidence (2 items)
Supports the claim: OpenAI,
GPT‑5 is significantly less likely to hallucinate than our previous models.
Why it matters here: OpenAI's comparative evaluations report lower factual-error rates rather than their elimination, alongside gains in other capabilities. This supports improvement with persistent qualitative failures, while weighing against a reading that error rates were unchanged.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Supports the claim: ARC Prize Foundation,
rules must be applied differently
Why it matters here: The benchmark authors document continuing contextual-rule failures and discuss progress on earlier ARC tasks. Their results support the coexistence of advances and unresolved reasoning limitations.
- Publisher
- ARC Prize Foundation
- Kind of source
- Benchmark result
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
That’s exactly what it means to hit a wall, and exactly the particular set of obstacles I described in my most notorious (and prescient) paper, in 2022. Real progress on some dimensions, but stuck in place on others.
- Where
- paragraph 84
- Published
- 2025-08-09
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-gpt-5-overdue-overhyped-and-underwhelming-19e767-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-gpt-5-overdue-overhyped-and-underwhelming-19e767-a1.