“It also, at this point, is pretty obvious that generative AI isn't going to do much more than it does today.”
Our reading, the sentence that was checked:
Generative AI will not become much more capable than it is today.
Assessment: Contradicted Settled before its deadline on 2026-10-29 Evidence as of
Where and when Ed Zitron wrote it
Where's Your Ed At, Reality Check, paragraph 118.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment wyea-reality-check-44888d-a2
We recorded 20 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-04-28. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- Settled before its deadline
- Deadline state on 2026-10-29
- Settled before its deadline (the release’s reference date)
From the speaker
- generative AI
- isn't going to do much more
- than it does today
Our interpretation
- Today means April 28, 2025.
- The plain reading is a ceiling on substantial additional capability, rather than a prediction about profitability.
- A large, independently verified improvement in solving demanding tasks can refute that ceiling without demonstrating general intelligence.
- No deadline or numerical definition of much more is supplied. Benchmark choice and the significance of a gain are evaluator judgments.
- Models with reasoning and task-specific scaffolding count as generative-AI systems; the claim does not restrict future systems to an unchanged architecture or compute budget.
Counts as supported if: A decisive and applicable limit establishes that generative-AI systems cannot substantially exceed April 2025 capabilities. A temporary absence of progress is insufficient for this deadline-free claim.
Counts as contradicted if: Independent evaluation demonstrates a substantial new capability or a large improvement in reliable completion of demanding tasks compared with April 2025 systems.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Contradicted
The dated evidence goes against what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- Settled before its deadline
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
On April 22, 2025, ARC Prize reported that neither o3-low nor o3-medium surpassed 3% on ARC-AGI-2 [e2]. On December 5, 2025, it reported verified scores of 37.6% for Opus 4.5 and 54% for Poetiq's solution built on Gemini 3 Pro [e3]. Under the contract's interpretation, this large increase in verified task completion contradicts the forecast of no substantial capability gains beyond April 2025 [e2, e3].
Limitations
- The results do not establish gains across all practical applications or demonstrate general intelligence.
- The evidence does not isolate contributions from model changes, scaffolding, compute or benchmark-domain exposure.
- The comparison does not establish performance at matched compute budgets.
- Treating this benchmark gain as sufficient to refute the claim is an evaluator interpretation.
Evidence (3 items)
Goes against the claim: ARC Prize Foundation,
54%
Why it matters here: The independently verified result demonstrates a large increase in successful abstract-reasoning task completion over the April baseline. The report also cautions that refinement harnesses are domain-specific and that benchmark-domain training may contribute to performance.
- Where
- Industry Progress
- Publisher
- ARC Prize Foundation
- Kind of source
- Benchmark result
- Describes events of
- Public from
Context only: ARC Prize Foundation,
single-digit percentage scores
Why it matters here: The benchmark developer documents low initial performance on novel abstract-reasoning tasks. This establishes a pre-publication baseline, not a later outcome.
- Where
- Announcements
- Publisher
- ARC Prize Foundation
- Kind of source
- Benchmark result
- Describes events of
- Public from
Context only: ARC Prize Foundation,
3%
Why it matters here: Immediately before the post, the evaluator reported low and medium o3 results below 3%; it identified o3-medium as its strongest tested publicly available model. High-reasoning runs lacked sufficient coverage and were excluded from reliable scoring.
- Publisher
- ARC Prize Foundation
- Kind of source
- Benchmark result
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Where's Your Ed At, Reality Check
It also, at this point, is pretty obvious that generative AI isn't going to do much more than it does today.
- Where
- paragraph 118
- Published
- 2025-04-28
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment wyea-reality-check-44888d-a2: Current. Outcome: Contradicted. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
HeldBy this site
The final-text check failed: rationale_supported_by_evidence; overclaims: The later system uses a specialized refinement harness and a different compute budget..
About assessment wyea-reality-check-44888d-a1, which was not published.