Pundit ClaimsAI claims by Gary Marcus and Ed Zitron, with the evidence

Gary Marcus Marcus on AI Published

OpenAI’s Waterloo? [with corrections]

We recorded 9 claims from this post. 4 were checked against evidence. 5 can’t be checked. 8 more are held. Read the post at its source.

Claims from this post (9)

In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.

  1. Assessment: Can’t be checkedSaid

    “But media coverage is not science, and braggadacio alone cannot yield AGI.”

    What we tested: Hype and bragging alone cannot produce AGI.

  2. Assessment: UnresolvedSaid

    “Indeed, although it did not get the press it deserved, recent AAAI survey showed that the vast majority of academic AI researchers doubted that scaling would get us all the way to AGI. They were right.”

    What we tested: A recent AAAI survey showed that the vast majority of academic AI researchers doubted that scaling would get us all the way to AGI, and they were right.

  3. Assessment: Can’t be checkedSaid

    “There is a real sense in which GPT-5 is starting to look Altman’s Waterloo.”

    What we tested: GPT-5 is starting to look like Altman's decisive defeat.

  4. Assessment: MixedSaid

    “Some customers even wanted the old models back. Almost nobody felt satisfied.”

    What we tested: Almost no one was satisfied with GPT-5, and some customers wanted the old models back.

    What held: Shows that users asked for GPT-4o back and that OpenAI reinstated it, which supports the 'old models back' part. (TechRepublic, )

    What did not: Several business customers were satisfied enough to make GPT-5 their default within a week, which contradicts 'almost nobody felt satisfied'. (CNBC, )

  5. Assessment: Can’t be checkedSaid

    “Two years ago, many treated Altman like an oracle; in the eyes of many, Altman now looks more like a snake oil salesman.”

    What we tested: Two years ago many people treated Altman like an oracle, and now, in the eyes of many, he looks more like a snake oil salesman.

  6. Assessment: Can’t be checkedSaid

    “And you can’t make AI better if you drown out the critics and keep pouring good money after bad.”

    What we tested: AI cannot be made better if you drown out the critics and keep pouring good money after bad.

  7. Assessment: UnresolvedSaid

    “But between the disappointments of GPT-5 and a new study from METR that shows that LLMs do markedly better on coding benchmarks than in real-word practice, I think it is safe to say that LLMs won’t lead the way.”

    What we tested: LLMs will not be the approach that leads the way to AGI.

  8. Assessment: SupportedSaid

    “But between the disappointments of GPT-5 and a new study from METR that shows that LLMs do markedly better on coding benchmarks than in real-word practice, I think it is safe to say that LLMs won’t lead the way.”

    What we tested: LLMs do markedly better on coding benchmarks than in real-world coding practice.

    What happened: The trial involved 16 developers and 246 real issues; allowing AI increased completion time by 19%. METR explicitly contrasts this practical slowdown with impressive benchmark performance, supporting the study-specific reading. (METR, )

  9. Assessment: Can’t be checkedSaid

    “As people begin to recognize that truth (the bitter lesson about limits of Sutton’s The Bitter Lesson), scientists will start to charts new paths. One of the new paths may even give rise to what we so desperately need: AI that we can trust.”

    What we tested: As the limits of scaling become recognized, scientists will pursue new approaches, one of which may produce trustworthy AI.

Held from this post (8)

These records did not pass every check, so nothing about them is published. Held is not a finding.