“The ridicule is of me; the failure is of the lords of deep learning to resolve what ails their AI.”
Our reading, the sentence that was checked:
The lords of deep learning have failed to resolve what ails their AI.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, 26 months of ridicule and failure, paragraph 1.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-26-months-of-ridicule-and-failure-9b97b9-a3
We recorded 5 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-06-01. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- over the 26 months covered by the post's title (qualifier, stated by the speaker)
From the speaker
- the failure is of the lords of deep learning
- to resolve what ails their AI
- 26 months later (since the 'Deep Learning is Hitting a Wall' essay)
- reliability is still a dream
- toxicity and hallucinations are still with us
Our interpretation
- 'What ails their AI' is read from the post's own list: unreliability, hallucinations and toxicity in leading deep-learning systems such as large language models.
- 'Failure to resolve' is read as: by the post date, leading developers had not eliminated these problems, so their flagship systems still made confident factual errors. Progress or partial reductions do not count as resolution.
- The separate claim that we are 'no closer' on alignment is not graded here. Only the narrower claim that the problems remained unresolved is graded.
- The target period runs from the March 2022 essay to the post date, 2024-06-01.
Counts as supported if: Developer statements or independent evaluations from the period showing that leading deep-learning systems still hallucinated or were unreliable as of mid-2024.
Counts as contradicted if: Credible evidence that by June 2024 leading developers had effectively eliminated hallucinations and unreliability in their flagship systems, such as evaluations showing near-zero factual error rates in normal use.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The claim covers the 26 months in the post's title. Under the contract, that period runs from 2022-03-10 to the post date, 2024-06-01. The claim is read narrowly: leading developers had not resolved the problems the post names, mainly unreliability and hallucinations. In March 2023, OpenAI said GPT-4 'still is not fully reliable' and 'hallucinates' facts (e1). On May 30, 2024, Google said that 'odd, inaccurate or unhelpful' AI Overviews 'certainly did show up' (e2). In May 2024, a Stanford study found that legal AI tools gave incorrect information more than 17% of the time (Lexis+ AI, Ask Practical Law AI). It found that Westlaw's AI-Assisted Research hallucinated more than 34% of the time (e3). OpenAI also reported that GPT-4 scored 40% higher than GPT-3.5 on its internal factuality evaluations. In the same passage, it called hallucinations 'still a real issue' (e4). That shows a reduction, and the contract does not count a reduction as resolution. The post's statements about alignment and about the technology being 'exactly as incorrigible' are not graded here.
Limitations
- 'What ails their AI' is not defined in the sentence. The grade uses the problems the post names, mainly hallucinations and reliability.
- None of the cited evidence addresses toxicity, which the post also names.
- OpenAI's statement (e1) dates from March 2023, partway through the period, not from the post date.
- The evidence includes some measures of improvement: OpenAI's internal factuality score (e4) and Stanford's finding that the legal tools made fewer errors than GPT-4 (e3). It does not track how much the problems changed over the whole period or across all leading systems.
- The grade does not cover the post's stronger statements that there was no progress or that alignment is no closer.
- Google said AI Overviews generally do not 'hallucinate' in the way other LLM products might. Its admission (e2) is about inaccurate answers, not hallucination as such.
Evidence (4 items)
Supports the claim: OpenAI,
Despite its capabilities, GPT‑4 has similar limitations as earlier GPT models. Most importantly, it still is not fully reliable (it “hallucinates” facts and makes reasoning errors).
Why it matters here: The leading lab says its flagship model, released within the period, still hallucinates and is not fully reliable.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Supports the claim: Google,
But some odd, inaccurate or unhelpful AI Overviews certainly did show up.
Why it matters here: Days before the post, Google admitted its new AI search feature gave inaccurate answers at launch. This shows reliability problems were still present.
- Publisher
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Supports the claim: Stanford HAI / RegLab,
even these bespoke legal AI tools still hallucinate an alarming amount of the time: the Lexis+ AI and Ask Practical Law AI systems produced incorrect information more than 17% of the time, while Westlaw’s AI-Assisted Research hallucinated more than 34% of the time.
Why it matters here: In May 2024, an independent test found that commercial tools built on LLMs and marketed as hallucination-free still hallucinated often.
- Publisher
- Stanford HAI / RegLab
- Kind of source
- Independent measurement
- Describes events of
- Public from
Context only: OpenAI,
While still a real issue, GPT‑4 significantly reduces hallucinations relative to previous models (which have themselves been improving with each iteration).
Why it matters here: This shows real progress in reducing hallucinations during the period. It does not show they were resolved; the same passage calls them 'still a real issue'.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, 26 months of ridicule and failure
The ridicule is of me; the failure is of the lords of deep learning to resolve what ails their AI.
- Where
- paragraph 1
- Published
- 2024-06-01
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-26-months-of-ridicule-and-failure-9b97b9-a3: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-26-months-of-ridicule-and-failure-9b97b9-a3.
HeldBy this site
The final-text check failed: conditions_shown, rationale_supported_by_evidence; overclaims: The evidence of progress (e4) would complicate those claims..
About assessment marcus-26-months-of-ridicule-and-failure-9b97b9-a2, which was not published.