“For all the daily claims of “exponential progress”, and for all the tens of billions that have been invested since, reliability is still a dream, and of course toxicity and hallucinations are still with us.”
Our reading, the sentence that was checked:
Despite tens of billions of dollars invested since 2022, deep learning systems are still unreliable and still produce toxicity and hallucinations.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, 26 months of ridicule and failure, paragraph 7.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-26-months-of-ridicule-and-failure-8dd634-a1
We recorded 4 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-06-01. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- despite tens of billions invested since 2022 (qualifier, stated by the speaker)
From the speaker
- tens of billions that have been invested since
- reliability is still a dream
- toxicity and hallucinations are still with us
- 26 months later
Our interpretation
- The ordinary reading is that broadly dependable operation remained unsolved and that toxic outputs and hallucinations persisted. It does not require every system to fail on every task or deny improvements.
- The surrounding discussion concerns contemporary generative AI, particularly language models. These systems are the assessment's scope; the claim is not treated as a statement about every application of deep learning.
- For the ongoing technical state, use the current calendar year through publication, January 1 to June 1, 2024. AI evaluations have no common quarterly reporting period.
- The investment phrase describes the industry-wide backdrop since approximately April 2022. It is not a conditional forecast or a claim about a named company's accounts, so contract metric_scope is null.
Counts as supported if: Contemporary evaluations document substantive reliability failures, hallucinations, and toxic output in prominent language models, despite documented industry investment in the tens of billions since 2022.
Counts as contradicted if: Contemporary, sufficiently broad evaluations establish dependable operation across the relevant uses and show that hallucination and toxic-output problems have been resolved. Strong performance on selected tasks alone is insufficient.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Contemporary evaluations document persistent hallucinations, failed task execution, and toxicity-handling failures. Stanford also documents investment in the tens of billions. This supports the ordinary reading that these problems remained unresolved. The same evaluations show useful capabilities and substantial differences between models; they do not support a stronger claim that no progress occurred or that no application was reliable.
Limitations
- The reliability phrase is rhetorical and supplies no threshold.
- Some tests deliberately challenge safeguards or use difficult tasks.
- The toxicity paper is a preprint, and its hallucination benchmark uses a restrictive task-based definition; the hallucination finding here rests on the separate AISI evaluation.
- Industry investment does not measure expenditure specifically directed at these problems.
Evidence (3 items)
Supports the claim: UK AI Safety Institute,
hallucinated constraints
Why it matters here: The May evaluation documents invented constraints and false claims of task completion, directly supporting continuing hallucinations and unreliable autonomous execution. The report also records successful shorter tasks and strong scientific-question performance, so the failures do not imply universal incompetence.
- Publisher
- UK AI Safety Institute
- Kind of source
- Independent measurement
- Describes events of
- Public from
Supports the claim: Innodata researchers, via arXiv,
cannot handle toxicity well
Why it matters here: This evaluation identifies continuing toxicity-handling failures in tested Mistral models. It also finds substantial differences between models and strong GPT results; those findings limit any reading that all systems were equally unreliable.
- Where
- Abstract
- Publisher
- Innodata researchers, via arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: Stanford Institute for Human-Centered Artificial Intelligence,
$25 billion
Why it matters here: Stanford's account of its investment research corroborates the tens-of-billions backdrop within the period since 2022. It measures private investment across generative AI, not money spent specifically on resolving reliability.
- Publisher
- Stanford Institute for Human-Centered Artificial Intelligence
- Kind of source
- Secondary summary
- Measures
- Global generative AI sector; no individual company, Private investment, USD billions, estimate, 2023-01-01 to 2023-12-31
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, 26 months of ridicule and failure
For all the daily claims of “exponential progress”, and for all the tens of billions that have been invested since, reliability is still a dream, and of course toxicity and hallucinations are still with us.
- Where
- paragraph 7
- Published
- 2024-06-01
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-26-months-of-ridicule-and-failure-8dd634-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-26-months-of-ridicule-and-failure-8dd634-a1.