“OpenAI failed to make a more efficient model for Microsoft late last year, and while GPT-5 is meant to be “materially better,” it isn’t obvious what “better” means when GPT-4 performs worse at some tasks than its predecessor.”
Our reading, the sentence that was checked:
OpenAI failed to build a more efficient model for Microsoft in late 2023, and GPT-4 performs worse than its predecessor at some tasks.
Assessment: Mixed Evidence as of
Where and when Ed Zitron wrote it
Where's Your Ed At, Expectations Versus Reality, paragraph 16.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment wyea-expectations-versus-reality-78badf-a1
We recorded 28 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-05-13. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- failed to make a more efficient model for Microsoft
- late last year
- GPT-4 performs worse at some tasks than its predecessor
Our interpretation
- The efficiency-project reference is read as Arrakis, the project identified by the article linked from the post.
- Late last year means late 2023, not the date on which an earlier failure was reported.
- The linked GPT-4 discussion concerns changes between GPT-4 versions. Assess documented task regressions without converting them into a claim of overall inferiority.
- The target year encloses the historical GPT-4 comparisons and the claimed late-2023 project event. The late-year qualifier is tested separately rather than discarded.
- GPT-5 being meant to be materially better is background attribution, not a measurable forecast in the supplied proposition.
Counts as supported if: Evidence establishes the identified efficiency project's failure in late 2023 and at least one valid task regression against the relevant predecessor.
Counts as contradicted if: Evidence establishes that the identified project failed outside the stated late-2023 period, succeeded instead, or that the claimed task regressions do not hold under the relevant comparison.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Mixed
Part of the claim holds and part does not, and the record says which part is which.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The reported Arrakis efficiency failure and task-specific GPT-4 regressions have support. The chronology does not: the accessible account places Arrakis's cancellation in mid-2023, while the reporting appeared in October. Thus the substance holds, but the literal late-last-year timing does not. This result depends on treating that phrase as the event date rather than the reporting date.
Limitations
- Arrakis evidence is secondary and traces to one underlying report.
- The timing distinction drives the mixed outcome.
- The benchmark supports particular version regressions, not overall GPT-4 inferiority.
Evidence (3 items)
Supports the claim: The Indian Express,
its performance was inefficient.
Why it matters here: This supports the substance of the reported efficiency-project failure. It is secondary reporting, not a primary project record or independent corroboration of The Information's sources.
- Where
- Article standfirst
- Publisher
- The Indian Express
- Kind of source
- News report
- Describes events of
- Public from
Supports the claim: arXiv; Lingjiao Chen, Matei Zaharia and James Zou,
more formatting mistakes
Why it matters here: The study documents a task-specific regression between GPT-4 versions under its evaluation procedure. Formatting and instruction-following regressions do not establish that GPT-4 became generally less capable.
- Where
- Abstract
- Publisher
- arXiv; Lingjiao Chen, Matei Zaharia and James Zou
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Goes against the claim: Synthedia,
scrapped in mid-2023
Why it matters here: The account places cancellation in the middle of 2023, contradicting the literal late-2023 timing while corroborating that a failure occurred. The year-only event date avoids inventing a precise cancellation month.
- Where
- Paragraph 2
- Publisher
- Synthedia
- Kind of source
- Secondary summary
- Describes events of
- Public from
What would settle it
Primary project records would strengthen the chronology. Clarifying whether late last year refers to cancellation or reporting would remove the interpretation-dependent part of the verdict.
Where it was said (1)
Said once in the sources this register covers.
: Where's Your Ed At, Expectations Versus Reality
OpenAI failed to make a more efficient model for Microsoft late last year, and while GPT-5 is meant to be “materially better,” it isn’t obvious what “better” means when GPT-4 performs worse at some tasks than its predecessor.
- Where
- paragraph 16
- Published
- 2024-05-13
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment wyea-expectations-versus-reality-78badf-a1: Current. Outcome: Mixed. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment wyea-expectations-versus-reality-78badf-a1.