“Their forte has been and always will be making shit up; they can’t for the life of them (speaking metaphorically of course) check facts.”
Our reading, the sentence that was checked:
For GPT models on their own, making things up has been and always will be their forte, and they cannot check facts.
Assessment: Unresolved No deadline stated on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, GPT-5 and irrational exuberance, paragraph 9.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-gpt-5-and-irrational-exuberance-187146-a1
We recorded 17 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2023-04-08. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- “has been and always will be”
- Deadline state when assessed
- No deadline stated
- Deadline state on 2026-10-29
- No deadline stated (the release’s reference date)
Conditions
- GPTs on their own (scope, stated by the speaker)
From the speaker
- Their forte has been and always will be making shit up
- they can’t for the life of them
- check facts
- GPT’s on their own
Our interpretation
- Always will be makes this an open-ended forecast with no deadline.
- The surrounding standalone qualification excludes retrieval and external verification tools. It is a scope restriction, not an antecedent that must later occur.
- The ordinary reading concerns dependable factual verification and characteristic fabrication, not an inability to answer any factual question correctly.
- Fact-checking requires distinguishing true from false claims against independently established ground truth. Answer generation and consistency checking are relevant but different measurements.
- No numerical reliability threshold is supplied; isolated successes and isolated failures cannot settle the broad claim.
Counts as supported if: Continuing factual unreliability can support the current limitation, but no finite series of evaluations establishes the assertion about every future GPT model.
Counts as contradicted if: Replicated evaluations establish dependable, broadly applicable fact-checking by standalone GPT models, contradicting the inability component, or establish that fabrication has ceased to be characteristic under representative use.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No deadline stated
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The evidence distinguishes partial factuality checking from dependable general verification. SelfCheckGPT challenges a literal claim of zero ability, while tool-free factuality benchmarks measure a related but different task. Neither settles the ordinary reading of the full forecast. No decisive early event was established, and continued errors cannot prove an assertion about all future GPT models.
Limitations
- A literal zero-ability reading would yield a different assessment of the fact-checking component.
- Factual recall, consistency detection and independent verification are different tasks.
- The cited evaluations do not exhaust every model available by the evaluation date.
Evidence (2 items)
Context only: arXiv; Google DeepMind researchers,
without tools
Why it matters here: SimpleQA Verified evaluates factual recall without tools and includes GPT models. Its results bear on factual reliability, but answering difficult factual questions is not a direct test of broad fact-checking ability and cannot settle an unlimited forecast.
- Where
- Section 4, Results
- Publisher
- arXiv; Google DeepMind researchers
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: arXiv; Potsawee Manakul, Adian Liusie and Mark J. F. Gales,
rank passages
Why it matters here: This provides material evidence against a literal assertion of no factuality-checking ability. However, it uses a sampling-based procedure and a limited evaluation, so it does not establish dependable general verification by a standalone GPT.
- Publisher
- arXiv; Potsawee Manakul, Adian Liusie and Mark J. F. Gales
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Replicated, broad evaluations showing dependable standalone verification with independent ground truth and no retrieval or external verification tools could falsify the inability component. The unlimited future assertion cannot be established by continued present-day failures.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, GPT-5 and irrational exuberance
Their forte has been and always will be making shit up; they can’t for the life of them (speaking metaphorically of course) check facts.
- Where
- paragraph 9
- Published
- 2023-04-08
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-gpt-5-and-irrational-exuberance-187146-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-gpt-5-and-irrational-exuberance-187146-a1.