“But we are kidding ourselves if it we think the solution is near at hand.”
Our reading, the sentence that was checked:
A solution to AI unreliability and hallucination is not near at hand.
Assessment: Supported Deadline passed on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, No, RAG is probably not going to rescue the current situation, paragraph 36.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-no-rag-is-probably-not-going-to-rescue-c82ea4-a1
We recorded 15 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-02-23. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- “near at hand”
- Deadline state when assessed
- Deadline passed
- Deadline state on 2026-10-29
- Deadline passed (the release’s reference date)
From the speaker
- the solution
- near at hand
- six-month project
- perfectly reliable, hallucination-free chatbots by this time next year
Our interpretation
- The solution means resolving the broad factual unreliability and hallucination problem discussed in the post.
- I interpret near at hand using the surrounding six-month and one-year statements, with the latest deadline being 2025-02-23.
- The ordinary reading predicts that broadly trustworthy chatbots will not arrive within that interval. It does not predict that reliability will fail to improve.
- Evidence of frequent material errors can support this reading without requiring a philosophical proof that error-free software is impossible.
Counts as supported if: Evaluations during the stated near-term interval, including evidence near its end, show that leading general-purpose or retrieval-enabled systems still make substantial factual errors and that no broadly validated solution has emerged.
Counts as contradicted if: By 2025-02-23, an available system or method demonstrably resolves broad chatbot unreliability, supported by independent evaluations across varied factual and reasoning tasks and realistic use.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- Deadline passed
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Under the post's six-to-twelve-month context, the prediction is supported. Research published before the one-year deadline documented substantial errors in leading assistants even with access to sources. The contemporary scientific assessment also found unresolved reliability problems. Legal RAG evaluations showed meaningful improvements, but continuing hallucinations. This supports the prediction that a broad solution was not imminent; it does not establish that RAG is useless or that a new architecture is necessary.
Limitations
- The one-year deadline is a contextual interpretation of near at hand.
- The BBC's 51% measure includes issues beyond factual errors and should not be described as a hallucination rate.
- The principal late-period tests concern December 2024 systems, not every system on 2025-02-23.
- The evidence supports a practical assessment of unresolved reliability, not proof that no private or specialized solution existed.
Evidence (3 items)
Supports the claim: BBC,
‘significant issues’ of some form
Why it matters here: The BBC tested ChatGPT, Copilot, Gemini and Perplexity with access to BBC sources. Beyond the broader issue measure quoted here, it reported factual errors in 19% of answers citing BBC content, supporting continued factual unreliability.
- Publisher
- BBC
- Kind of source
- Independent measurement
- Describes events of
- Public from
Supports the claim: International AI Safety Report,
Existing guardrails to contain and mitigate reliability issues are not fail-proof.
Why it matters here: The scientific review reports continuing reliability problems and limitations in methods for mitigating hallucinations. The event date uses its stated writing cutoff rather than treating January publication as a new experiment.
- Publisher
- International AI Safety Report
- Kind of source
- Secondary summary
- Describes events of
- Public from
Context only: arXiv; Varun Magesh and coauthors,
reduce hallucinations
Why it matters here: The study found that legal RAG tools reduced hallucinations relative to GPT-4, while still producing substantial errors. This qualifies any suggestion that RAG offered no benefit, but does not demonstrate a general solution.
- Publisher
- arXiv; Varun Magesh and coauthors
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, No, RAG is probably not going to rescue the current situation
But we are kidding ourselves if it we think the solution is near at hand.
- Where
- paragraph 36
- Published
- 2024-02-23
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-no-rag-is-probably-not-going-to-rescue-c82ea4-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-no-rag-is-probably-not-going-to-rescue-c82ea4-a1.