“The models aren't going to get better.”
Our reading, the sentence that was checked:
Even with racks of Blackwell GPUs, generative AI models are not going to get better.
Assessment: Contradicted Settled before its deadline on 2026-10-29 Evidence as of
Where and when Ed Zitron wrote it
Where's Your Ed At, Godot Isn't Making it, paragraph 59.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment wyea-godot-isnt-making-it-ef7b25-a1
We recorded 33 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-12-03. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- Settled before its deadline
- Deadline state on 2026-10-29
- Settled before its deadline (the release’s reference date)
Conditions
- even with Blackwell GPUs (qualifier, stated by the speaker)
From the speaker
- Microsoft or Google has racks-worth of Blackwell GPUs
- The models aren't going to get better.
Our interpretation
- The ordinary reading is a prediction of no further material model-capability improvement, including after Blackwell becomes available.
- Preserve the hardware qualification: evidence must identify models developed using Blackwell infrastructure.
- Comparable scores on established capability tests can demonstrate improvement. The claim does not require equal training expenditure or equal inference budgets.
- The surrounding discussion permits a narrower reading about pre-training alone. That reading is not identical to the categorical sentence being assessed.
Counts as supported if: A finite period without improvement would be consistent with the forecast but would not establish indefinite stagnation.
Counts as contradicted if: A documented Blackwell-developed model demonstrates material improvement over capabilities publicly demonstrated before the post on an established, substantially comparable capability evaluation.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Contradicted
The dated evidence goes against what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- Settled before its deadline
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Under the ordinary capability reading, the categorical prediction is contradicted. Microsoft's Blackwell-trained MAI-Thinking-1 reports 84.2% on GPQA Diamond, above the pre-post GPT-4o, o1-preview, and research o1 results on that benchmark. The report identifies both training hardware and scoring methods. This establishes a capability gain under published settings, not that Blackwell alone caused it. A narrower claim that pre-training alone would stop improving requires a separate assessment.
Limitations
- Company-reported evaluations, without independent replication of this comparison.
- Inference budgets and grading procedures differ.
- The result concerns demonstrated benchmark capability, not improvement on every task.
- The outcome depends on reading the categorical sentence as covering overall model capability.
Evidence (3 items)
Supports the claim: Microsoft AI,
16k GB200 GPUs
Why it matters here: Microsoft documents actual large-scale Blackwell training, establishing that the hardware scenario occurred. The MAI-Thinking-1 report also identifies earlier training on an Azure GB200 cluster.
- Publisher
- Microsoft AI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Goes against the claim: Microsoft AI,
averaged over 16 rollouts.
Why it matters here: The report identifies MAI-Thinking-1's base-model training on 8K GB200 GPUs and reports 84.2% GPQA Diamond pass@1, exceeding the pre-post results in e3. It uses the same 198-question Diamond subset; its large output budget and grading procedure limit exact comparability.
- Publisher
- Microsoft AI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: OpenAI,
o1 improved over the state-of-the-art.
Why it matters here: Appendix A reports GPQA Diamond pass@1 of 50.6% for GPT-4o, 73.3% for o1-preview, and 77.3% for the research o1 model. These establish a pre-post baseline, including the stronger research model rather than only the released preview.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Where's Your Ed At, Godot Isn't Making it
The models aren't going to get better.
- Where
- paragraph 59
- Published
- 2024-12-03
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment wyea-godot-isnt-making-it-ef7b25-a1: Current. Outcome: Contradicted. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.