Pundit ClaimsAI claims by Gary Marcus and Ed Zitron, with the evidence

Claims checked against evidence, page 20 of 31

Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.

Claims checked against evidence (3100)

Each of these claims was checked against dated evidence.

Page 20 of 31: claims 1901 to 2000 of 3100, ordered by the date first said, newest first. To find a claim by its words, use search.

  1. Assessment: UnresolvedSaid

  2. Assessment: UnresolvedSaid

  3. Assessment: UnresolvedSaid

  4. Assessment: SupportedSaid

    Gary Marcus

    “LLMs are already dangerous, despite their limits.”

    What happened: The researchers demonstrated unfavorable job assignments and harsher hypothetical criminal judgments associated with African American English. This establishes a concrete discriminatory risk, while the experiment does not establish that employers or courts actually imposed these outcomes. (arXiv, )

  5. Assessment: UnresolvedSaid

  6. Assessment: UnresolvedSaid

  7. Assessment: ContradictedSaid

    Ed Zitron

    “The products aren't coming.”

    What happened: The disclosure dates general availability to May 2025 and reports continued growth in weekly users and business subscriptions in February 2026. Its reported commercial scale supplies evidence that adoption extended beyond an experimental launch. (Anthropic, )

  8. Assessment: ContradictedSaid

    Ed Zitron

    “I also want to be clear that none of these companies ever had a plan.”

    What happened: Microsoft specified concrete generative AI functions inside Word and other work applications. That is a product and deployment strategy beyond purchasing GPUs, regardless of whether the promised benefits were achieved. (Microsoft, )

  9. Assessment: SupportedSaid

    Ed Zitron

    “OpenAI currently spends $2.35 to make $1, and Microsoft CFO Amy Hood said that OpenAI would cut into Microsoft's profits this October, losing it a remarkable $1.5 billion, "mainly because of an expected loss from OpenAI" according to CNBC.”

    What happened: These estimates imply $8.7 billion of expenses and a ratio rounding to $2.35. CNBC also reports exclusions, including equity-based compensation, so this supports the cited estimate rather than comprehensive accounting expenses. (CNBC, )

  10. Assessment: UnresolvedSaid

  11. Assessment: UnresolvedSaid

  12. Assessment: UnresolvedSaid

  13. Assessment: UnresolvedSaid

  14. Assessment: UnresolvedSaid

  15. Assessment: UnresolvedSaid

  16. Assessment: UnresolvedSaid

  17. Assessment: UnresolvedSaid

  18. Assessment: ContradictedSaid

    Ed Zitron

    “The entire tech industry has become oriented around a dead-end technology that requires burning billions of dollars to provide inessential products that cost them more money to serve than anybody would ever pay.”

    What happened: The preceding sentence identifies 2024 revenue of $4 billion and inference compute and hosting costs of $2.4 billion. This reported positive serving margin supplies a counterexample to the categorical assertion, without establishing company-wide profitability. (Epoch AI, )

  19. Assessment: UnresolvedSaid

  20. Assessment: UnresolvedSaid

  21. Assessment: UnresolvedSaid

  22. Assessment: UnresolvedSaid

  23. Assessment: UnresolvedSaid

  24. Assessment: UnresolvedSaid

  25. Assessment: ContradictedSaid

    Ed Zitron

    “Generative AI's products have effectively been trapped in amber for over a year.”

    What happened: Developers could use the API to have Claude inspect screens, move a cursor, click and type. This materially extends possible workflows, although Anthropic describes the release as experimental and error-prone. (Anthropic, )

  26. Assessment: UnresolvedSaid

  27. Assessment: MixedSaid

    Ed Zitron

    “The hallucination problem is one that is nowhere closer to being solved — and, at least with the current technology — may never go away, and it makes it a non-starter for a great many business tasks, where you need a high level of reliability.”

    What held: Testing on May 23–27 found frequent hallucinations in a specialized legal research product. This supports the barrier to unverified, high-reliability legal work under the stated interpretation, but does not establish the number of business tasks excluded. (arXiv, )

    What did not: Table 2 reports SimpleQA hallucination rates of 0.61 for GPT-4o and 0.44 for o1-preview, with improvements on the other listed tests. The same paragraph reports contrary anecdotal feedback and cautions against generalizing beyond these evaluations. (OpenAI, )

  28. Assessment: ContradictedSaid

    Ed Zitron

    “The models aren't going to get better.”

    What happened: The report identifies MAI-Thinking-1's base-model training on 8K GB200 GPUs and reports 84.2% GPQA Diamond pass@1, exceeding the pre-post results in e3. It uses the same 198-question Diamond subset; its large output budget and grading procedure limit exact comparability. (Microsoft AI, )

  29. Assessment: UnresolvedSaid

  30. Assessment: UnresolvedSaid

  31. Assessment: UnresolvedSaid

  32. Assessment: SupportedSaid

    Gary Marcus

    “Two years later Google’s search share hasn’t diminished all that much.”

    What happened: This supplies the launch-era worldwide baseline. Together with e2, it supports a modest decline; historical primary figures were not retrievable through the page reader. (Local Dialog, )

  33. Assessment: UnresolvedSaid

  34. Assessment: UnresolvedSaid

  35. Assessment: UnresolvedSaid

  36. Assessment: UnresolvedSaid

  37. Assessment: SupportedSaid

    Gary Marcus

    “GPT-4 was better than GPT-3 in almost every way; o1 is a far less uniform improvement.”

    What happened: The study evaluates legacy GPT-3 models and GPT-4 under aligned settings across more than twenty benchmarks and seven capability categories. Its overall analysis reports broad improvements across those categories from GPT-3 to GPT-4. (Zheng et al., ByteDance and University of Illinois Urbana-Champaign, arXiv, )

  38. Assessment: UnresolvedSaid

  39. Assessment: SupportedSaid

    Gary Marcus

    “(In some cases o1 doesn’t even do better than GPT-4).”

    What happened: The stated count implies that o1 did not outperform GPT-4o in the remaining three subcategories. It supports the limited 'some cases' claim under the contextual reading of GPT-4. (OpenAI, )

  40. Assessment: UnresolvedSaid

  41. Assessment: UnresolvedSaid

  42. Assessment: UnresolvedSaid

  43. Assessment: SupportedSaid

    Gary Marcus

    “Many other Tech CEOs and others, including Marc Andreeseen (A16Z), Alexandr Wang (ScaleAI), have also used the phrase “hitting a wall” and the idea of diminishing returns in the last couple weeks.”

    What happened: Wang publicly used the 'wall' framing on Nov 20, 2024, three days before the post. He partly agreed that a wall had been hit, and the report links it to diminishing returns from training. (The Verge, )

  44. Assessment: UnresolvedSaid

  45. Assessment: SupportedSaid

    Gary Marcus

    “Scaling was always a hypothesis.”

    What happened: Early scaling research built its claims from empirical measurements across several tasks. This supports the distinction between observed regularities and guarantees about their indefinite continuation. (arXiv; Hestness et al., )

  46. Assessment: UnresolvedSaid

  47. Assessment: SupportedSaid

    Gary Marcus

    “They are merely empirical generalizations that held for a certain period time, when there was enough fresh data and compute.”

    What happened: The authors explicitly characterize their loss relationships as empirical. Their reported measurements cover finite ranges of model size, data and training compute. (arXiv; Kaplan et al., )

  48. Assessment: UnresolvedSaid

  49. Assessment: UnresolvedSaid

  50. Assessment: UnresolvedSaid

  51. Assessment: UnresolvedSaid

  52. Assessment: UnresolvedSaid

  53. Assessment: SupportedSaid

    Gary Marcus

    “Many of us, including Melanie Mitchell, Subbarao Kambahapati, Emily Bender, Ernie Davis, etc. have been emphasizing for ages that there are principled limits with LLMs.”

    What happened: Bender and Koller's paper makes an explicit conceptual-limit argument more than four years before the post. The metadata gives July 2020; July 31 is used as a conservative availability bound. (Association for Computational Linguistics, )

  54. Assessment: UnresolvedSaid

  55. Assessment: SupportedSaid

    Gary Marcus

    “People aren’t ignoring GenAI; they are waiting to see if it will work.”

    What happened: A third of the firms were still testing. That fits 'waiting to see if it will work.' (CNBC (syndicated by NBC San Diego), )

  56. Assessment: SupportedSaid

    Gary Marcus

    “But only 25% of the correspondents thought it was worth it.”

    What happened: Matches his 25% yes, 25% no, 50% too-soon breakdown exactly. (CNBC (syndicated by NBC San Diego), )

  57. Assessment: UnresolvedSaid

  58. Assessment: UnresolvedSaid

  59. Assessment: SupportedSaid

    Gary Marcus

    “(is OpenAI’s o1 100 times better than GPT4? Certainly not by their own data)”

    What happened: On AIME, one of OpenAI's best showcases for o1, the gain is about sixfold (74% vs 12%), far below 100x. (OpenAI, )

  60. Assessment: UnresolvedSaid

  61. Assessment: UnresolvedSaid

  62. Assessment: SupportedSaid

    Gary Marcus

    “It’s just fantasy to imagine selling 200 times as many humanoid robots in the nearish term when nobody knows how to build a single safe, reliable, generally useful humanoid right now, at any price.””

    What happened: The accompanying expert discussion says simple tasks still require substantial training and performance remains strongest in controlled, repetitive settings. This supports the ordinary reading that dependable general usefulness had not been established. (IEEE Transmitter, )

  63. Assessment: UnresolvedSaid

  64. Assessment: UnresolvedSaid

  65. Assessment: UnresolvedSaid

  66. Assessment: UnresolvedSaid

  67. Assessment: ContradictedSaid

    Gary Marcus

    “The difference between GPT-3 and GPT-4 was by all accounts massive; the difference between GPT-4 and o1 is by all accounts modest.”

    What happened: The company reported substantial preference gains across three important reasoning domains for the actual preview model. This challenges the asserted agreement about only modest improvement, while remaining a company-run evaluation. (OpenAI, )

  68. Assessment: UnresolvedSaid

  69. Assessment: SupportedSaid

    Gary Marcus

    “Jailbreaks aren’t new, but even after years of them, the tech industry has nothing like a robust response.”

    What happened: The study evaluates four defenses and reports substantial vulnerability to human multi-turn attacks despite strong results against automated single-turn attacks. This supports the ordinary reading that established defenses lacked broad robustness. (arXiv; Scale AI and UC Berkeley researchers, )

  70. Assessment: UnresolvedSaid

  71. Assessment: UnresolvedSaid

  72. Assessment: SupportedSaid

    Ed Zitron

    “It isn’t. You saw a prototype and heard a speech.”

    What happened: Explicitly identifies the unveiled Cybercab as a prototype. The article's headline uses arrival language, but its description supports Zitron's distinction. (The Verge, )

  73. Assessment: UnresolvedSaid

  74. Assessment: UnresolvedSaid

  75. Assessment: UnresolvedSaid

  76. Assessment: MixedSaid

    Ed Zitron

    “far too many simply ran exactly what Musk said and acted as if he’d proven that these were actual robots walking around and doing things on their own accord, as opposed to what was blatantly obvious: that they were controlled remotely by human operators.”

    What held: Reports a bartender robot's acknowledgment of assistance, alongside accounts of teleoperated guest interactions. (Ars Technica, )

    What did not: Contradicts including walking in a blanket account of remote control. The reported operation combined autonomous locomotion with human-assisted interactions. (Ars Technica, )

  77. Assessment: SupportedSaid

    Gary Marcus

    “Fall off on increasingly large multiplication problems has repeatedly been observed, both in older models and newer models.”

    What happened: For older models (GPT-3, ChatGPT, GPT-4), multi-digit multiplication accuracy falls from near perfect to zero as problem size grows. (arXiv (Dziri et al., NeurIPS 2023), )

  78. Assessment: MixedSaid

    Gary Marcus

    “I strongly believe the current results are robust.”

    What held: An independent team built a controlled benchmark with irrelevant context (IC) and found LLMs significantly sensitive to it. This reproduces the core GSM-NoOp-style finding. (arXiv (Yang et al.), )

    What did not: A peer-reviewed re-analysis found the GSM-Symbolic drop significant for only 8 of 20 models, and partly explained by larger numbers in the new dataset. That cuts against the robustness of that headline result. (arXiv (Długosz, Oliveira, Díaz-Rodríguez; accepted EMNLP 2026), )

  79. Assessment: SupportedSaid

    Gary Marcus

    “Performance is ok on small problems, but quickly tails off.”

    What happened: o1-preview scored 97.8% on small problems but 23.63% on 6-20 block problems, and its correct answers on the larger set were mostly the shorter plans. That is good performance on small problems with a quick fall-off. (arXiv (Valmeekam, Stechly, Kambhampati), )

  80. Assessment: SupportedSaid

    Gary Marcus

    “Failure to follow the rules of chess is another continuing failure of formal reasoning:”

    What happened: On a 1,000-puzzle benchmark in mid-2024, the strongest model tested (GPT-4o) made illegal moves 12.7% of the time, and most other LLMs made more illegal moves than legal ones. That matches a continuing failure to follow chess rules before the post. (The Register, )

  81. Assessment: UnresolvedSaid

  82. Assessment: ContradictedSaid

    Gary Marcus

    “We have been told that AI will “solve physics” (per Sam Altman), make major breakthroughs in medicine, battery technology, etc, but so far none of that has materialized.”

    What happened: The paper reports demonstrated accuracy in the CASP14 assessment conducted in May–July 2020. Together with e2, it establishes an achieved major research advance rather than an untested promise. (Nature, )

  83. Assessment: UnresolvedSaid

  84. Assessment: UnresolvedSaid

  85. Assessment: UnresolvedSaid

  86. Assessment: UnresolvedSaid

  87. Assessment: ContradictedSaid

    Ed Zitron

    “That product is extremely commoditized. The free version of ChatGPT is effectively identical to the free version of Anthropic's Claude, Meta's AI assistant, Microsoft's Copilot, and even Twitter's "Grok."”

    What happened: The launch announcement places access in premium subscription tiers during the target quarter. The explicit premium-only history in e2 corroborates that this was not an ordinary free consumer offering. (xAI, )

  88. Assessment: UnresolvedSaid

  89. Assessment: UnresolvedSaid

  90. Assessment: UnresolvedSaid

  91. Assessment: UnresolvedSaid

  92. Assessment: UnresolvedSaid

  93. Assessment: SupportedSaid

    Ed Zitron

    “All of this is happening at a time when an astronomical amount of talent is leaving the company — the talent that OpenAI desperately needs to build products that people will actually pay for.”

    What happened: The simultaneous departure announcements cover the chief technology officer, chief research officer and a research vice president. Those roles support the ordinary reading of a substantial concentration of relevant leadership departures. (The Register, )

  94. Assessment: ContradictedSaid

    Ed Zitron

    “To hit a revenue target of $11.6 billion in 2025, OpenAI would need to increase revenue from ChatGPT customers by 310%.”

    What happened: The derived counterexample reaches the target with a 213.5% increase in the stipulated ChatGPT baseline, assuming the stated mix and a full year at the resulting rate. This is a calculation from reported premises, not a measured revenue outcome. (CNBC, )

  95. Assessment: SupportedSaid

    Ed Zitron

    “To be abundantly clear, as it stands, OpenAI currently spends $2.35 to make $1.”

    What happened: The reported estimates imply $8.7 billion of expenses on their loss basis and an expense-to-revenue ratio of 2.35135. This supports the arithmetic reading, not a claim about realized expenditure. (CNBC, )

  96. Assessment: UnresolvedSaid

  97. Assessment: SupportedSaid

    Ed Zitron

    “OpenAI's products are increasingly-commoditized, with Google, Meta, Amazon and even Microsoft building generative AI models to compete.”

    What happened: The retrospective comparison identifies capability convergence by Llama 3.1, released during the target quarter. It supports increasing model substitutability while also documenting historical lags behind closed models. (Epoch AI, )

  98. Assessment: UnresolvedSaid

  99. Assessment: UnresolvedSaid

  100. Assessment: UnresolvedSaid