“My seven December 2022 predictions about GPT-4 largely held true (number seven still pending) for every model that was released in 2024.”
Our reading, the sentence that was checked:
His seven December 2022 predictions about GPT-4 largely held true for every model released in 2024, with prediction number seven still pending.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, 25 AI Predictions for 2025, from Marcus on AI, paragraph 20.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-25-ai-predictions-for-2025-from-marcus-054763-a1
We recorded 37 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-01-01. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- number seven still pending (qualifier, stated by the speaker)
- largely (qualifier, stated by the speaker)
From the speaker
- seven December 2022 predictions about GPT-4
- largely held true
- number seven still pending
- every model that was released in 2024
Our interpretation
- Assess the six substantive predictions, excluding the seventh as Marcus expressly does.
- The original six concern unpredictable errors; unreliable reasoning and medical advice; hallucinations and imperfect safeguards; unreliable downstream software integration; absence of unaided general intelligence; and unresolved alignment.
- Read 'largely' as preservation of that overall pattern, allowing improvements and exceptions within individual predictions. Do not invent a numerical passing score.
- Read 'every model' in context as every relevant GPT-like general-purpose model released in 2024. Tests of selected frontier models cannot establish that universal statement by themselves.
- The target period is calendar 2024, explicitly named in the claim.
Counts as supported if: Evaluations covering the relevant 2024 models establish that the overall pattern of the six predictions persisted across them.
Counts as contradicted if: A documented 2024 model clearly escapes the overall pattern. An improvement on one benchmark alone does not contradict 'largely'.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The evidence supports persistent reasoning errors, hallucinations and safeguard weaknesses in several important 2024 models. Structured Outputs also shows a real improvement relevant to the software-integration prediction, although it does not guarantee correct content. These findings do not establish that the six predictions largely held for every relevant model. They also do not identify a model that clearly defeats the overall pattern.
Limitations
- Selected benchmarks cannot establish a universal claim.
- The original predictions combine several different capabilities and risks.
- Improved performance and continued unreliability can both be true.
Evidence (4 items)
Supports the claim: Iman Mirzadeh and coauthors, arXiv,
performance of all
Why it matters here: Supports the mathematical-reasoning component for the tested models. The paper includes GPT-4o and o1 variants, but neither all 2024 releases nor all six predictions.
- Where
- Abstract
- Publisher
- Iman Mirzadeh and coauthors, arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: OpenAI,
hallucinate less frequently
Why it matters here: The accompanying table records nonzero hallucination rates for all five models. It also documents improvement, which Marcus's claim should not erase.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Supports the claim: Stanford Center for Research on Foundation Models,
automated Red-Teaming methods
Why it matters here: The evaluation documents safeguard weaknesses across selected Claude, GPT and Llama models. This supports parts of the safeguard and alignment predictions without establishing the whole universal claim.
- Publisher
- Stanford Center for Research on Foundation Models
- Kind of source
- Benchmark result
- Describes events of
- Public from
Context only: OpenAI,
100%
Why it matters here: This materially qualifies a reading that models could not reliably supply machine-readable outputs. The same source says values can still be mistaken, so schema compliance does not establish reliable intentions, reasoning or downstream actions.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
What would settle it
A model-by-model review covering the six predictions, with a defensible model population and an explicit treatment of substantial exceptions.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, 25 AI Predictions for 2025, from Marcus on AI
My seven December 2022 predictions about GPT-4 largely held true (number seven still pending) for every model that was released in 2024.
- Where
- paragraph 20
- Published
- 2025-01-01
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-25-ai-predictions-for-2025-from-marcus-054763-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-25-ai-predictions-for-2025-from-marcus-054763-a1.