“Imagining this as a six-month project is naive.”
Our reading, the sentence that was checked:
Reliable, hallucination-free AI via RAG will not be achieved within six months.
Assessment: Supported Deadline passed on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, No, RAG is probably not going to rescue the current situation, paragraph 30.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-no-rag-is-probably-not-going-to-rescue-c4dc42-a1
We recorded 15 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-02-23. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- “six-month”
- Deadline state when assessed
- Deadline passed
- Deadline state on 2026-10-29
- Deadline passed (the release’s reference date)
From the speaker
- Imagining this as a six-month project is naive.
- To reliably address all queries
- not just some obvious ones
Our interpretation
- Read the sentence in context as predicting that broadly reliable RAG query answering would not be achieved within six months of publication.
- The deadline is August 23, 2024.
- The standard concerns broad factual query answering. Success on a restricted dataset or a system that avoids answering most questions does not meet it.
- Assess the publicly demonstrated state of the technology, using broad evaluations and scrutiny of claimed breakthroughs.
Counts as supported if: By the deadline, representative broad evaluations and leading demonstrated RAG systems still show material factual failures or inability to answer substantial portions of the query set, with no verified breakthrough meeting the stated broad scope.
Counts as contradicted if: A RAG system available by August 23, 2024 demonstrates reliable, essentially hallucination-free answering across the broad range of queries described, with reproducible evaluation beyond a restricted dataset.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- Deadline passed
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The public evidence supports the forecast's broad reading. CRAG found substantial shortcomings in industry RAG systems, and the July competition results remained far from complete query answering. Acurai announced hallucination elimination before the deadline, but its validation covered a restricted dataset and input conditions. That result does not establish the broad reliability Marcus described.
Limitations
- This is an assessment of demonstrated technology, not proof that no undisclosed system existed.
- The broad evaluations precede the deadline; they are not an exact August 23 snapshot.
- The evidence does not establish Marcus's separate assertion that reliable RAG requires solving AGI.
Evidence (4 items)
Supports the claim: Yang et al., arXiv,
63% questions
Why it matters here: CRAG tests diverse factual questions across domains and question types. Its reported industry-system performance falls well short of broadly reliable query answering; the remaining percentage should not automatically be treated as a hallucination rate.
- Where
- Abstract, version 1
- Publisher
- Yang et al., arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: Meta and AIcrowd,
28.40%
Why it matters here: The winning task scores were 28.40%, 42.7% and 47.8%, showing that broad RAG challenge performance remained incomplete shortly before the deadline. These are competition scores, not percentages of hallucinated answers.
- Publisher
- Meta and AIcrowd
- Kind of source
- Benchmark result
- Describes events of
- Public from
Context only: Acurai Inc., distributed through ACCESSWIRE,
RAGTruth Corpus.
Why it matters here: Acurai announced a hallucination-free service before the deadline. The stated validation concerned RAGTruth, so the announcement is material counterevidence but does not establish reliable answering across Marcus's broader scope.
- Publisher
- Acurai Inc., distributed through ACCESSWIRE
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: Wood and Forbes, arXiv,
three passages.
Why it matters here: The later paper reports a sample of 37 and limits its finding to factual input passages and small contexts. This clarifies why the company's result does not demonstrate the broad capability at issue; it is not treated as an August outcome.
- Publisher
- Wood and Forbes, arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, No, RAG is probably not going to rescue the current situation
Imagining this as a six-month project is naive.
- Where
- paragraph 30
- Published
- 2024-02-23
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-no-rag-is-probably-not-going-to-rescue-c4dc42-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-no-rag-is-probably-not-going-to-rescue-c4dc42-a1.