GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.
We recorded 30 claims from this post. 18 were checked against evidence. 12 can’t be checked. 14 more are held. Read the post at its source.
Claims from this post (30)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
Observation
“What the mainstream media mostly hasn’t told you yet is that a few days later, hardly anybody is buying Altman’s story.”
Our reading: A few days after launch, hardly anybody believes Altman's claims that GPT-5 is like a PhD-level expert.
No horizon Evidence as of
Limitation: The evidence is news reports of individual reactions, not representative data.
Assessment: UnresolvedSaid
Causal claim
“Because expectations were through the roof, a huge number of people viewed GPT 5 as a major letdown. By the end of the night, OpenAI’s street cred had dramatically fallen.”
Our reading: Sky-high expectations caused many to see GPT-5 as a major letdown, and OpenAI's credibility fell dramatically on launch night.
No horizon Evidence as of
Limitation: The market movement is documented through news reporting rather than verified original records.
Assessment: SupportedSaid
Observation
“And, indeed, much as I anticipated here two weeks ago, the problems I have been pointing out over the last quarter century still lingered.”
Our reading: GPT-5 still has the longstanding problems Marcus has identified, such as chess rule-following and world models.
No horizon Evidence as of
Limitation: The main evidence is a single demonstration, not a benchmark.
Assessment: UnresolvedSaid
Observation
“The challenge of parts and wholes in generative images that Ernest Davis and I discussed here in December fared no better.”
Our reading: GPT-5 shows no improvement on the parts-and-wholes problem in generated images.
No horizon Evidence as of
Limitation: The only systematic test predates GPT-5 and used ChatGPT-4o.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“For all that, GPT-5 is not a terrible model.”
Our reading: GPT-5 is not a terrible model.
No horizon
Assessment: SupportedSaid
Observation
“People had grown to expect miracles, but GPT-5 is just the latest incremental advance.”
Our reading: GPT-5 is merely an incremental advance, not a radical leap.
No horizon Evidence as of
Limitation: The result depends on reading the comparison as against the immediately preceding models.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Certainly, in a rational world, their valuation would take a hit.”
Our reading: In a rational world, OpenAI's valuation would certainly take a hit.
No horizon
Qualifier: in a rational world
Assessment: Can’t be checkedSaid
Forecast
“GPT-5 is unlikely to be ahead of the pack for more than a couple months.”
Our reading: GPT-5 is unlikely to remain ahead of competing models for more than a couple of months.
No horizon
Assessment: SupportedSaid
Observation
“Many of their best people have left.”
Our reading: Many of OpenAI's best people have left the company.
No horizon Evidence as of
Limitation: Documented prominence does not establish an objective ranking of OpenAI's best employees.
Assessment: UnresolvedSaid
Observation
“Elon is moving faster. Anthropic and Google and many others are nipping at their heels. Their relationship with Microsoft has frayed.”
Our reading: Elon Musk is moving faster than OpenAI, Anthropic and Google and many others are nipping at OpenAI's heels, and OpenAI's relationship with Microsoft has frayed.
No horizon Evidence as of
Limitation: Release cadence, benchmark rank and rate of capability improvement are different measures.
Assessment: Can’t be checkedSaid
Observation
“People are wising up that LLMs are not in fact AGI-adjacent.”
Our reading: LLMs are not close to AGI, and people are increasingly realizing this.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“By rights, Altman’s reputation should by now be completely burned.”
Our reading: Sam Altman's reputation should by now be completely burned given his AGI claims.
No horizon
Assessment: Can’t be checkedSaid
Rhetoric
“In hindsight, that was all bullshit.”
Our reading: In hindsight, Altman's statements (his joke that AGI had been achieved internally, his claim that OpenAI knows how to build AGI, and his claim that talking to GPT-5 is like talking to a PhD-level expert) were all bullshit.
No horizon
Qualifier: in hindsight
Assessment: Can’t be checkedSaid
Causal claim
“And the worst part? Altman brought it all on himself. Had he not kept hinting at the moon, people might have been fine with just another incremental update.”
Our reading: Altman's overhyping caused the GPT-5 backlash; without it people might have accepted an incremental update.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“He may not even be the right CEO for OpenAI anymore:”
Our reading: Sam Altman may no longer be the right CEO for OpenAI.
No horizon
Assessment: UnresolvedSaid
Observation
“It’s not like other systems are faring much better.”
Our reading: Other generative AI systems such as Grok are not faring much better than GPT-5.
No horizon Evidence as of
Limitation: The packet contains no verified primary comparison from the August 7–9 launch window.
Assessment: UnresolvedSaid
Observation
“The real news is a breaking study from Arizona State University that fully vindicates what I have told you for nearly 30 years—and more recently what Apple told you—about the core weakness of LLMs: their inability to generalize broadly.”
Our reading: LLMs' core weakness is their inability to generalize broadly, as confirmed by an ASU study.
No horizon Evidence as of
Limitation: This assesses the August 5 version available when Marcus wrote.
Assessment: ContradictedSaid
Observation
“The ASU team showed that exactly the same thing was true even in the latest, greatest models. Throw in every gadget invented since 1998, and the Achilles’ Heel I identified then still remains.”
Our reading: The ASU study showed that even the latest, greatest models cannot reliably extend universals outside their training space of examples, so the Achilles' heel Marcus identified in 1998 still remains despite every gadget invented since.
No horizon Evidence as of
Limitation: This does not establish that frontier models generalize reliably.
Assessment: Can’t be checkedSaid
Rhetoric
“We have been fed a steady diet of bullshit for the last several years.”
Our reading: Over the last several years, we have been fed a steady diet of bullshit about AI.
No horizon
Assessment: UnresolvedSaid
Observation
“Allegedly godlike models that turn out to be incremental advances.”
Our reading: Models hyped as godlike turn out to be incremental advances.
No horizon Evidence as of
Limitation: A smaller aggregate benchmark gain can coexist with substantial gains in particular uses.
Assessment: UnresolvedSaid
Observation
“Promises for world-changing science that rarely materialize.”
Our reading: Promises that AI will produce world-changing science rarely materialize.
No horizon Evidence as of
Limitation: The searches were not an exhaustive census of AI science promises.
Assessment: UnresolvedSaid
Observation
“Driverless cars that still are only available in couple percent of the world’s cities.”
Our reading: Driverless cars are available in only a couple percent of the world's cities.
No horizon Evidence as of
Limitation: This does not establish that worldwide availability exceeded his estimate.
Assessment: MixedSaid
Observation
“Cherry-picked studies, benchmark-gaming, and now even vibe-coded graphs, with zero transparency about how systems work or how they have been trained; public science is in the rear view mirror.”
Our reading: AI has been marked by cherry-picked studies, benchmark-gaming and now even vibe-coded graphs, with zero transparency about how systems work or how they have been trained, and public science is in the rear view mirror.
No horizon Evidence as of
Limitation: The judgment depends on reading zero transparency as no substantive disclosure, rather than insufficient disclosure for reproduction.
Assessment: UnresolvedSaid
Observation
“Even some of the tech bros are waking up to the reality that “AGI in 2027” was marketing, not reality.”
Our reading: Even some tech bros are waking up to the reality that 'AGI in 2027' was marketing, not reality.
No horizon Evidence as of
Limitation: The observation concerns public attitudes by August 2025, not whether AGI ultimately arrives in 2027.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Nobody with intellectual integrity should still believe that pure scaling will get us to AGI.”
Our reading: No intellectually honest person should still believe pure scaling will reach AGI.
No horizon
Assessment: UnresolvedSaid
Observation
“Dozens of shots on goal (Grok, Claude, Gemini) etc have invariably faced the same problems. Distribution shift has never been solved.”
Our reading: Dozens of shots on goal, such as Grok, Claude and Gemini, have invariably faced the same problems, and distribution shift has never been solved.
No horizon Evidence as of
Limitation: Support for recurring weaknesses is stronger than support for the universal wording.
Assessment: SupportedSaid
Observation
“That’s exactly what it means to hit a wall, and exactly the particular set of obstacles I described in my most notorious (and prescient) paper, in 2022. Real progress on some dimensions, but stuck in place on others.”
Our reading: LLM development has hit a wall: real progress on some dimensions but stuck on others.
No horizon Evidence as of
Limitation: The outcome depends on reading stuck as persistence of qualitative failures, as the surrounding passage indicates.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“After half a trillion dollars in that direction, it is obviously time to move on. The disappointing performance of GPT-5 should make that enormously clear.”
Our reading: After half a trillion dollars spent on scaling, the field should move on from the pure-scaling hypothesis.
No horizon
Assessment: UnresolvedSaid
Forecast
“Pure scaling simply isn’t the path to AGI.”
Our reading: Pure scaling simply isn't the path to AGI and will not get us there.
No deadline stated Evidence as of
Limitation: Benchmark achievement is narrower than general intelligence.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“All I am saying is give neurosymbolic AI with explicit world models a chance.”
Our reading: The field should give neurosymbolic AI with explicit world models a chance.
No horizon
Held from this post (14)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: OpenAI is not profitable.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: Within hours of release GPT-5 showed ridiculous errors and hallucinations, the same pattern as earlier models.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: The dominant reaction to the GPT-5 launch, apart from a few influencers, was major disappointment.
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: Public skepticism about OpenAI and Sam Altman is increasing.
- The two reviews reached different assessments. Our reading, not assessed: OpenAI is being forced to cut prices.
- The two reviews reached different assessments. Our reading, not assessed: General purpose AI agents have turned out to suck so badly that people struggle to find real-world use cases for them.
- The two reviews reached different assessments. Our reading, not assessed: Many of OpenAI's best people who have left did so to start competitors.
- The two reviews read the claim differently
- Attribution is uncertain
- The two reviews reached different assessments. Our reading, not assessed: GPT-5 arrived late and was underwhelming.