Pundit ClaimsAI claims by Gary Marcus and Ed Zitron, with the evidence

Changes, page 10 of 55

Every substantive change to the register, newest first. Speaker events record what the people covered said or changed. Site events record what this site published, corrected, flagged, or withdrew. The two never share a label.

Substantive changes (10984)

Page 10 of 55: changes 1801 to 2000 of 10984, newest first. Dates are when the change was recorded.

  1. HeldBy this siteGary Marcus

    Held record marcus-six-or-seven-predictions-for-ai-2026-8befdb

    e1 is dated 2025, which is not inside the period described (2025-01-01 to 2025-12-20); it can only be context; e2 is dated 2025, which is not inside the period described (2025-01-01 to 2025-12-20); it can only be context

  2. HeldBy this siteGary Marcus

    Held record marcus-sir-demis-hassabis-vs-sir-demis-hassabis-f89b93

    hedged language is not scored, even on a plain reading

  4. Assessment publishedBy this siteGary Marcus

    As of December 2022, LLMs hallucinated and their errors could be hard to catch.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-shame-on-google-twice-ccd9e6-a2.

  5. HeldBy this siteGary Marcus

    Held record marcus-shame-on-google-twice-655a77

    The final-text check failed: citations_correct.

  6. Assessment publishedBy this siteGary Marcus

    LLMs cannot be trusted to run algorithms as problem complexity and distance from the training distribution grow.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-seven-replies-to-the-viral-apple-c8a2fd-a1.

  8. Assessment publishedBy this siteGary Marcus

    We can't know in advance which model, if any, is big enough for any given problem.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-seven-replies-to-the-viral-apple-4722b8-a2.

  9. Assessment publishedBy this siteGary Marcus

    The most popular rebuttal to the Apple reasoning paper was full of errors.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-seven-replies-to-the-viral-apple-121b14-a1.

  10. HeldBy this siteGary Marcus

    Held record marcus-seven-replies-to-the-viral-apple-055cba

    The final-text check failed: proposition_faithful.

  11. Assessment publishedBy this siteGary Marcus

    There is already pressure from GenAI to keep coal plants running.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-seven-reasons-why-the-world-should-8fb2a8-a1.

  12. Assessment publishedBy this siteGary Marcus

    OpenAI has never disclosed the water consumption figures for GPT-4.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-seven-reasons-why-the-world-should-023ed2-a1.

  13. Assessment publishedBy this siteGary Marcus

    Google has access to data, especially from search and email, at a scale no other company has.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-seven-reasons-i-wouldnt-count-google-dca8a9-a2.

  14. Assessment publishedBy this siteGary Marcus

    The LLM market will end up as a multi-way tie with relatively modest profits.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-seven-reasons-i-wouldnt-count-google-8bc68d-2-a1.

  15. Assessment publishedBy this siteGary Marcus

    Google/Alphabet is hemorrhaging talent, its morale is poor, and its latest model is delayed.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-seven-reasons-i-wouldnt-count-google-11635a-a2.

  16. Assessment publishedBy this siteGary Marcus

    Only a few companies can match Google's compute and distributed-systems expertise.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-seven-reasons-i-wouldnt-count-google-025cba-a1.

  17. HeldBy this siteGary Marcus

    Held record marcus-seven-lies-in-four-sentences-ac13d1

    The anthropic review found the criteria unfair to the speaker's words.

  18. Assessment publishedBy this siteGary Marcus

    OpenAI is under SEC investigation as of late February 2024.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-serious-medical-error-from-perplexitys-cd39c9-a2.

  20. Assessment publishedBy this siteGary Marcus

    Perplexity's GenAI search engine gave a serious medical error when asked about sitting at a computer after open-heart surgery.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-serious-medical-error-from-perplexitys-73e917-a2.

  21. HeldBy this siteGary Marcus

    Held record marcus-sentient-ai-for-the-love-of-darwin-522232

    e1 is dated 2023-08, which is not inside the period described (2023-08-01 to 2023-08-21); it can only be context; e2 is dated 2023-08, which is not inside the period described (2023-08-01 to 2023-08-21); it can only be context

  22. HeldBy this siteGary Marcus

    Held record marcus-scott-alexanders-misleading-victory-ab464f

    e3 is dated 2025, which is not inside the period described (2025-01-01 to 2025-07-09); it can only be context

  23. HeldBy this siteGary Marcus

    Held record marcus-scott-alexanders-misleading-victory-95f3b1

    The openai review reads the claim differently.

  24. Assessment publishedBy this siteGary Marcus

    The AI field is full of Band-Aid solutions that work for one problem but not the next.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-scott-alexanders-misleading-victory-776723-a1.

  25. Assessment publishedBy this siteGary Marcus

    GPT-4o failed to accurately diagnose errors in its own image outputs.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-scott-alexanders-misleading-victory-0df80e-a1.

  26. HeldBy this siteGary Marcus

    Held record marcus-scoop-what-former-employees-of-openai-47d2a3

    The openai review found the criteria unfair to the speaker's words.

  27. Assessment publishedBy this siteGary Marcus

    There will be many cases of people trusting and promoting ChatGPT outputs that are flat-out wrong.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-scientists-please-dont-let-your-chatbots-ec90d4-a1.

  28. Assessment publishedBy this siteGary Marcus

    ChatGPT got a pace unit conversion wrong and lacks reliability, though it looks precise.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-scientists-please-dont-let-your-chatbots-dfbd25-a1.

  29. Assessment publishedBy this siteGary Marcus

    People are starting to treat ChatGPT as a credentialed scientific collaborator.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-scientists-please-dont-let-your-chatbots-c65d93-a1.

  30. HeldBy this siteGary Marcus

    Statistical word prediction cannot substitute for real mathematical computation.

    The final-text check failed: rationale_supported_by_evidence, citations_correct; overclaims: The January 31, 2023 evaluation does not establish when its tested responses were collected, so it cannot establish performance during the November 30, 2022–January 14, 2023 target period [e2]..

  31. HeldBy this siteGary Marcus

    Held record marcus-scientists-governments-and-corporations-d7d41d

    The anthropic review does not accept e1 as supports.

  32. HeldBy this siteGary Marcus

    Held record marcus-scarjo-is-just-the-tip-of-the-iceberg-1d01d4

    The anthropic review found the criteria unfair to the speaker's words.

  34. HeldBy this siteGary Marcus

    Held record marcus-scaling-is-over-the-bubble-may-be-d72c8e

    The final-text check failed: rationale_supported_by_evidence; overclaims: OpenAI reported a 95% reduction in per-token pricing since GPT-4's launch on January 31, 2025 [e3]. | This qualifies the characterization of the release as straightforward confirmation of industry stagnation..

  35. HeldBy this siteGary Marcus

    Held record marcus-scaling-is-over-the-bubble-may-be-83fee8

    The anthropic review does not accept e2 as contradicts.

  36. Assessment publishedBy this siteGary Marcus

    As of April 2025, GPT-5 has not been released.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-scaling-is-over-the-bubble-may-be-3bf91c-a1.

  37. Assessment publishedBy this siteGary Marcus

    Nvidia's stock fell by about a third in 2025 through early April, far more than the overall market.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-scaling-is-over-the-bubble-may-be-39c944-a1.

  39. Assessment publishedBy this siteGary Marcus

    None of the known problems discussed in Marcus's thread has a principled solution thus far.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-sam-altman-thinks-that-agi-is-basically-97196d-a1.

  40. Assessment publishedBy this siteGary Marcus

    LLMs generalize well to similar items but are unreliable in unfamiliar regimes, and minor variations sometimes break them.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-sam-altman-thinks-that-agi-is-basically-95ab84-a1.

  41. Assessment publishedBy this siteGary Marcus

    Commonsense reasoning in current AI systems remains unreliable, as it long has been.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-sam-altman-thinks-that-agi-is-basically-6e0341-a1.

  42. Assessment publishedBy this siteGary Marcus

    The problem of distribution shift in neural models continues to be unsolved.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-sam-altman-thinks-that-agi-is-basically-467fc6-a1.

  43. Assessment publishedBy this siteGary Marcus

    A recent paper showed o1's performance drops about 30% on Putnam math problems with minor variations like changed variable names.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-sam-altman-thinks-that-agi-is-basically-3a4d5f-a2.

  44. HeldBy this siteGary Marcus

    Held record marcus-sam-altman-and-the-day-nvidias-meteoric-f74bb5

    The openai review found the criteria unfair to the speaker's words.

  45. HeldBy this siteGary Marcus

    Held record marcus-sam-altman-and-the-day-nvidias-meteoric-e48f07

    The openai review reads the claim differently.

  46. HeldBy this siteGary Marcus

    Held record marcus-sam-altman-and-the-day-nvidias-meteoric-ddd246

    The final-text check failed: rationale_supported_by_evidence; overclaims: No consistent, split-adjusted price series covers November 2022 through February 2026; the reported prices cannot establish the full-period trend..

  47. Assessment publishedBy this siteGary Marcus

    Nvidia's rise rested on the mistaken idea that scaling would lead to AGI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-sam-altman-and-the-day-nvidias-meteoric-bfa8ba-a1.

  48. HeldBy this siteGary Marcus

    Held record marcus-sam-altman-and-the-day-nvidias-meteoric-448a96

    The anthropic review does not accept e1 as supports.

  49. Assessment publishedBy this siteGary Marcus

    Even state-of-the-art AI systems can reliably perform only a small fraction of tasks in typical human occupations.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-rumors-of-agis-arrival-have-been-981d53-a1.

  51. Assessment publishedBy this siteGary Marcus

    Optimizing for benchmarks produces systems that perform well on tests but degrade in modestly different real-world settings.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-rumors-of-agis-arrival-have-been-6f5c93-a2.

  52. HeldBy this siteGary Marcus

    Held record marcus-rethinking-driverless-cars-dee4f9

    The final-text check failed: citations_correct.

  54. Assessment publishedBy this siteGary Marcus

    Uber and Lyft can stop worrying about being disintermediated by machines, because they will still need human drivers for quite some time.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-rethinking-driverless-cars-7ff76b-a1.

  56. HeldBy this siteGary Marcus

    Held record marcus-rethinking-driverless-cars-6cd8fe

    the contract's accounting basis is unknown; a financial claim is scored only on a stated basis; supported needs at least one supporting item of outcome evidence

  57. Assessment publishedBy this siteGary Marcus

    Personal cars moonlighting as self-driving taxis will not happen this decade, and maybe not next decade either.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-rethinking-driverless-cars-12ff54-a2.

  58. HeldBy this siteGary Marcus

    Held record marcus-reports-of-llms-mastering-math-have-e59e20

    The final-text check failed: citations_correct.

  59. Assessment publishedBy this siteGary Marcus

    DeepMind's neurosymbolic AlphaProof and AlphaGeometry are immune to producing confidently incorrect proofs.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-reports-of-llms-mastering-math-have-dd731d-a1.

  61. Assessment publishedBy this siteGary Marcus

    On the 2025 USAMO, none of the leading reasoning LLMs tested scored higher than 5% overall.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-reports-of-llms-mastering-math-have-552698-a1.

  62. HeldBy this siteGary Marcus

    Held record marcus-reports-of-llms-mastering-math-have-37ac50

    The openai review found the criteria unfair to the speaker's words.

  63. HeldBy this siteGary Marcus

    Held record marcus-reports-of-llms-mastering-math-have-3352e6

    The final-text check failed: citations_correct.

  64. Assessment publishedBy this siteGary Marcus

    LLMs sometimes falsely claim to have solved problems, reflecting a failure to sanity-check their own work.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-reports-of-llms-mastering-math-have-2bc2e7-a1.

  65. Assessment publishedBy this siteGary Marcus

    The o3 proof of USAMO 2025 problem 2 posted by Nestler was invalid, corroborating the report.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-reports-of-llms-mastering-math-have-05b28b-a1.

  66. HeldBy this siteGary Marcus

    Held record marcus-psa-microsoft-may-be-training-on-d9447c

    hedged language is not scored, even on a plain reading

  67. Assessment publishedBy this siteGary Marcus

    More companies will follow an opt-out playbook to collect training data, hiding it in obscure settings and Terms of Service.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-psa-microsoft-may-be-training-on-930bb2-2-a2.

  68. Assessment publishedBy this siteGary Marcus

    Covert opt-out scraping of user data for AI training will happen a lot.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-psa-microsoft-may-be-training-on-0d52b0-a2.

  69. Assessment publishedBy this siteGary Marcus

    Oracle does not have the chips it would need to fulfill the contracts in the $300 billion OpenAI deal under discussion, or even the cash to buy those chips.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-peak-bubble-9a2c00-a2.

  70. Assessment publishedBy this siteGary Marcus

    GPT-5 was not tantamount to AGI, even though OpenAI thought or claimed it would be.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-peak-bubble-60174f-a1.

  72. HeldBy this siteGary Marcus

    Held record marcus-partial-regurgitation-and-how-llms-e2d174

    The final-text check failed: rationale_supported_by_evidence, citations_correct; overclaims: The distinctive ingredient, quantity and purpose support describing the answer as a close reworking of that source [e5, e6]. | Ars independently reports tracing the glue answer to Reddit and reproducing its illustrated errors..

  74. HeldBy this siteGary Marcus

    Held record marcus-partial-regurgitation-and-how-llms-12a6ed

    The anthropic review reads the claim differently.

  75. Assessment publishedBy this siteGary Marcus

    LLMs understand their output only superficially and can easily be tricked, e.g., by prompt injection.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openclaw-aka-moltbot-is-everywhere-a00750-a1.

  76. Assessment publishedBy this siteGary Marcus

    OpenClaw will, like AutoGPT, report having completed tasks it has not actually completed.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openclaw-aka-moltbot-is-everywhere-69869e-a1.

  77. HeldBy this siteGary Marcus

    Held record marcus-openclaw-aka-moltbot-is-everywhere-5f1672

    The anthropic review found the criteria unfair to the speaker's words.

  78. Assessment publishedBy this siteGary Marcus

    OpenClaw inherits the prompt-injection and security weaknesses of LLM agents.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openclaw-aka-moltbot-is-everywhere-5db4ae-a1.

  80. Assessment publishedBy this siteGary Marcus

    Moltbook has already been attacked.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openclaw-aka-moltbot-is-everywhere-1c0361-a1.

  81. Assessment publishedBy this siteGary Marcus

    John Schulman, Andrej Karpathy, Jan Leike and perhaps a dozen others have left OpenAI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openais-slow-motion-train-wreck-dd6d38-a1.

  82. HeldBy this siteGary Marcus

    Held record marcus-openais-slow-motion-train-wreck-da1797

    The final-text check failed: citations_correct.

  84. Assessment publishedBy this siteGary Marcus

    OpenAI had an operating loss of $5 billion last year.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openais-slow-motion-train-wreck-9bd962-a2.

  85. HeldBy this siteGary Marcus

    Held record marcus-openais-slow-motion-train-wreck-5dd5f5

    e7 measures something else (period 2024-09-11..2024-09-11 is not 2024-09-01..2024-09-25); it can only be context unless it is derived with stated inputs

  86. Assessment publishedBy this siteGary Marcus

    OpenAI faces many pending lawsuits.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openais-slow-motion-train-wreck-2adeb2-a1.

  87. HeldBy this siteGary Marcus

    Held record marcus-openais-slow-motion-train-wreck-0f42d1

    The two reviews differ: anthropic historical_observation/not_applicable/supported, openai historical_observation/not_applicable/unresolved.

  88. HeldBy this siteGary Marcus

    Held record marcus-openais-o3-and-tyler-cowens-misguided-e05476

    The final-text check failed: proposition_faithful.

  89. HeldBy this siteGary Marcus

    Held record marcus-openais-o3-and-tyler-cowens-misguided-cfde32

    The two reviews differ: anthropic causal_assessment/not_applicable/mixed, openai causal_assessment/not_applicable/unresolved.

  90. Assessment publishedBy this siteGary Marcus

    Scaling of AI models is no longer working as effectively as it once did.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openais-o3-and-tyler-cowens-misguided-b9c3c2-a2.

  91. HeldBy this siteGary Marcus

    Held record marcus-openais-o3-and-tyler-cowens-misguided-ae375b

    The two reviews differ: anthropic historical_observation/not_applicable/mixed, openai historical_observation/not_applicable/contradicted.

  92. Assessment publishedBy this siteGary Marcus

    o3 works only some of the time and is markedly better on some problems than others, as Marcus predicted in November 2024.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openais-o3-and-tyler-cowens-misguided-9a20fa-a1.

  93. Assessment publishedBy this siteGary Marcus

    LLMs have a massive problem with fabricating citations.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openais-o3-and-tyler-cowens-misguided-80c113-a1.

  95. Assessment publishedBy this siteGary Marcus

    OpenAI's 2015 openness pledge was a recruiting promise its leaders did not intend to keep, while letting the public believe it was open.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openais-lies-and-half-truths-9ee796-a1.

  96. Assessment publishedBy this siteGary Marcus

    Despite Greg Brockman's 2020 promise, OpenAI has never given Marcus advance access to red-team any of its software.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openais-lies-and-half-truths-67464f-a1.

  97. HeldBy this siteGary Marcus

    Held record marcus-openais-lies-and-half-truths-671370

    e2 is dated 2024-03, which is not inside the period described (2015-01-01 to 2024-03-15); it can only be context

  98. Assessment publishedBy this siteGary Marcus

    Many companies are trying ChatGPT without putting it into full-time production.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openais-got-99-problems-and-twitch-848fce-a1.

  99. Assessment publishedBy this siteGary Marcus

    OpenAI's unusual relationship with Microsoft is sure to receive considerable scrutiny.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openais-got-99-problems-and-twitch-829d86-a1.

  100. Assessment publishedBy this siteGary Marcus

    OpenAI's candor as a company will remain an issue.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openais-got-99-problems-and-twitch-506fa2-a1.

  101. Assessment publishedBy this siteGary Marcus

    LLMs sometimes leak private data, an unresolved problem, and hallucinations remain problematic.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openais-got-99-problems-and-twitch-0200e7-a1.

  102. HeldBy this siteGary Marcus

    Held record marcus-openais-disconcerting-hack-of-huggingface-ea2fc5

    The openai review reads the claim differently.

  103. HeldBy this siteGary Marcus

    Held record marcus-openais-disconcerting-hack-of-huggingface-9073cf

    The openai review does not accept e2 as supports.

  104. HeldBy this siteGary Marcus

    Held record marcus-openais-disconcerting-hack-of-huggingface-40c010

    The final-text check failed: citations_correct.

  105. Assessment publishedBy this siteGary Marcus

    OpenAI had privileged access to FrontierMath data that competitors lacked and did not acknowledge it.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openais-dirty-december-o3-demo-doesnt-fc328e-a1.

  106. HeldBy this siteGary Marcus

    Held record marcus-openais-dirty-december-o3-demo-doesnt-b44e4e

    e1 is dated 2025-04, which is not inside the period described (2025-03-01 to 2025-04-23); it can only be context

  107. Assessment publishedBy this siteGary Marcus

    OpenAI's December 2024 o3 demo had experimental design problems and misleading graphs that omitted competing work.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openais-dirty-december-o3-demo-doesnt-460523-a1.

  108. Assessment publishedBy this siteGary Marcus

    OpenAI's reported 75% o3 score on ARC-AGI does not readily replicate.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openais-dirty-december-o3-demo-doesnt-2f37cd-a1.

  109. HeldBy this siteGary Marcus

    Held record marcus-openais-code-red-c894c1

    The openai review reads the claim differently.

  110. Assessment publishedBy this siteGary Marcus

    If Masayoshi Son backs out of investing tens of billions, OpenAI will have serious cash problems.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openai-in-deep-trouble-f1c417-a1.

  111. Assessment publishedBy this siteGary Marcus

    GPT-4.5 is very expensive.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openai-in-deep-trouble-c77544-a1.

  112. Assessment publishedBy this siteGary Marcus

    OpenAI's high burn rate, perhaps a couple billion dollars a month, leaves it limited runway despite last year's financing.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openai-in-deep-trouble-b7d52a-a1.

  113. Assessment publishedBy this siteGary Marcus

    OpenAI and partners do not have all or most of the Stargate funding in hand.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openai-in-deep-trouble-b72563-a1.

  114. HeldBy this siteGary Marcus

    Held record marcus-openai-in-deep-trouble-a59cf9

    The anthropic review reads the claim differently.

  115. Assessment publishedBy this siteGary Marcus

    DeepSeek triggered a price war that reduces OpenAI's potential profits.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openai-in-deep-trouble-9af59d-a1.

  116. Assessment publishedBy this siteGary Marcus

    Whatever lead OpenAI had two years ago has been squandered.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openai-in-deep-trouble-918e99-a1.

  117. Assessment publishedBy this siteGary Marcus

    Some departed OpenAI staff have founded serious competitors with similar IP.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openai-in-deep-trouble-6a40a6-a1.

  118. Assessment publishedBy this siteGary Marcus

    Scaling has not brought OpenAI or anyone else to AGI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-openai-in-deep-trouble-52753c-a1.

  119. HeldBy this siteGary Marcus

    Held record marcus-openai-in-deep-trouble-3e8845

    The anthropic review found the criteria unfair to the speaker's words.

  120. HeldBy this siteGary Marcus

    Many top people have left OpenAI.

    The final-text check failed: citations_correct.

  121. HeldBy this siteGary Marcus

    Held record marcus-openai-in-deep-trouble-02523c

    hedged language is not scored, even on a plain reading

  123. HeldBy this siteGary Marcus

    Held record marcus-open-source-is-not-the-same-as-open-e51b26

    The openai review reads the claim differently.

  124. HeldBy this siteGary Marcus

    Held record marcus-open-source-is-not-the-same-as-open-d283df

    The anthropic review reads the claim differently.

  125. HeldBy this siteGary Marcus

    Held record marcus-open-source-is-not-the-same-as-open-bc25ad

    The anthropic review does not accept e1 as contradicts.

  126. Assessment publishedBy this siteGary Marcus

    Open-weight models get open-source's good press but offer few of its advantages, being neither fully customizable nor transparent.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-open-source-is-not-the-same-as-open-98ab44-a1.

  127. Assessment publishedBy this siteGary Marcus

    Open-weight models can be run and post-trained but cannot be customized, improved or investigated nearly as much as open-source models.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-open-source-is-not-the-same-as-open-038fbb-a1.

  128. HeldBy this siteGary Marcus

    Held record marcus-oops-how-google-bombed-while-doing-c61755

    The two reviews differ: anthropic causal_assessment/not_applicable/mixed, openai causal_assessment/not_applicable/unresolved.

  129. Assessment publishedBy this siteGary Marcus

    Near-plagiarism results from generative systems reconstructing data fragments in statistically probable ways without any mechanism to assess originality.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-one-genai-image-ten-errors-f09bb8-a1.

  130. Assessment publishedBy this siteGary Marcus

    Hallucinations result from generative systems breaking data into fragments and reconstructing them without a fact-checking mechanism to assess coherence.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-one-genai-image-ten-errors-ce7373-a1.

  131. HeldBy this siteGary Marcus

    Held record marcus-one-genai-image-ten-errors-9a822a

    The final-text check failed: citations_correct.

  132. Assessment publishedBy this siteGary Marcus

    The Midjourney-created image discussed has at least ten ways in which it fails to match the prompt or real-world expectations.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-one-genai-image-ten-errors-6f376d-a1.

  133. HeldBy this siteGary Marcus

    Although every company is investing in AI, most are so far not getting huge returns from it.

    The final-text check failed: citations_correct.

  134. HeldBy this siteGary Marcus

    Held record marcus-on-employment-dont-panic-yet-0302c7

    The two reviews differ: anthropic historical_observation/not_applicable/supported, openai historical_observation/not_applicable/unresolved.

  135. HeldBy this siteGary Marcus

    Held record marcus-o3-agi-the-art-of-the-demo-and-what-fbe6ea

    The openai review found the criteria unfair to the speaker's words.

  136. Assessment publishedBy this siteGary Marcus

    Passing the ARC test is certainly not sufficient for AGI, though it may be necessary.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-o3-agi-the-art-of-the-demo-and-what-cf7529-a1.

  138. HeldBy this siteGary Marcus

    Held record marcus-o3-agi-the-art-of-the-demo-and-what-59e6e4

    The final-text check failed: citations_correct.

  139. Assessment publishedBy this siteGary Marcus

    On the morning of 2025-01-27, Nvidia stock was down more than 10% in pre-market trading.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-nvidia-could-soon-take-a-serious-3eb1b3-a1.

  140. HeldBy this siteGary Marcus

    Held record marcus-not-consistently-candid-485d62

    The final-text check failed: citations_correct.

  141. Assessment publishedBy this siteGary Marcus

    When you actually look at the data in the cited Turing Test study, humans are still ahead of the AI systems tested.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-nope-the-turing-test-has-not-been-e5d345-a1.

  142. Assessment publishedBy this siteGary Marcus

    In the cited Turing Test study, the 1960s ELIZA chatbot outperformed GPT-3.5.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-nope-the-turing-test-has-not-been-dd3f9e-a1.

  143. Assessment publishedBy this siteGary Marcus

    As of December 2023, no one, Google's Gemini included, was close to passing the Comprehension Challenge.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-nope-the-turing-test-has-not-been-9086b0-a1.

  144. Assessment publishedBy this siteGary Marcus

    As of December 2023, no AI software could meet Marcus's Comprehension Challenge.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-nope-the-turing-test-has-not-been-81898b-a1.

  145. Assessment publishedBy this siteGary Marcus

    The Eugene Goostman chatbot, celebrated in 2014, made no lasting contribution to AI.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-nonsense-on-stilts-87d145-a1.

  146. Assessment publishedBy this siteGary Marcus

    Software like LaMDA does not connect to the world; it is just autocomplete predicting which words fit a context.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-nonsense-on-stilts-52d256-a1.

  147. Assessment publishedBy this siteGary Marcus

    The AI community wishes the Turing Test would go away, and its members all know that beating the test is not meaningful.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-nonsense-on-stilts-4f4543-a1.

  148. Assessment publishedBy this siteGary Marcus

    Nearly all of the AI community considers the idea that LaMDA might be sentient ridiculous.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-nonsense-on-stilts-29cb82-a1.

  149. Assessment publishedBy this siteGary Marcus

    In 2026, AI will still not be choosing its own research goals in place of human scientists.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-nobel-prizes-and-the-ai-hype-hall-612ee6-a1.

  150. HeldBy this siteGary Marcus

    Held record marcus-noam-chomsky-and-gpt-3-e5d653

    The final-text check failed: citations_correct.

  151. HeldBy this siteGary Marcus

    Held record marcus-noam-chomsky-and-gpt-3-c8b259

    e3 is dated 2022, which is not inside the period described (2021-01-01 to 2022-05-21); it can only be context

  152. HeldBy this siteGary Marcus

    Held record marcus-noam-chomsky-and-gpt-3-9d2404

    The anthropic review does not accept e1 as supports.

  153. HeldBy this siteGary Marcus

    Held record marcus-noam-chomsky-and-gpt-3-9c77a9

    The final-text check failed: rationale_supported_by_evidence, citations_correct; overclaims: Those results establish progress on other evaluations without eliminating the documented Gopher shortfalls [e1, e3]..

  154. HeldBy this siteGary Marcus

    Held record marcus-noam-chomsky-and-gpt-3-2a95db

    The final-text check failed: citations_correct.

  155. Assessment publishedBy this siteGary Marcus

    Machines often solve benchmarks for the wrong reasons and building good benchmarks is an unsolved problem.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-virginia-agi-is-not-imminent-6ab6b8-a1.

  156. Assessment publishedBy this siteGary Marcus

    Machines are not close to exceeding typical humans at coping with unusual circumstances while driving.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-virginia-agi-is-not-imminent-43e8b7-a1.

  157. HeldBy this siteGary Marcus

    Held record marcus-no-sora-has-not-learned-physics-7f11db

    The openai review reads the claim differently.

  158. Assessment publishedBy this siteGary Marcus

    Because Sora is not a physics engine, a large fraction of its videos are inconsistent with physics or biology in basic ways.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-sora-has-not-learned-physics-4b17c9-a1.

  159. HeldBy this siteGary Marcus

    Held record marcus-no-sora-has-not-learned-physics-1fb3c2

    The final-text check failed: citations_correct; overclaims: e2 explanation: "The researchers found strong geometric consistency in ten publicly available Sora clips compared with competing generators." | e2 explanation: "This qualifies a broad dismissal of visual quality, but selected demos and a geometry measure cannot establish overall visual reliability.".

  160. Assessment publishedBy this siteGary Marcus

    A solution to AI unreliability and hallucination is not near at hand.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-rag-is-probably-not-going-to-rescue-c82ea4-a1.

  161. Assessment publishedBy this siteGary Marcus

    Reliable, hallucination-free AI via RAG will not be achieved within six months.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-rag-is-probably-not-going-to-rescue-c4dc42-a1.

  162. Assessment publishedBy this siteGary Marcus

    AI will eventually work reliably.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-rag-is-probably-not-going-to-rescue-9eb5c1-a2.

  163. Assessment publishedBy this siteGary Marcus

    Perfectly reliable, hallucination-free chatbots will not exist by February 2025.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-rag-is-probably-not-going-to-rescue-559937-a2.

  164. HeldBy this siteGary Marcus

    Held record marcus-no-need-to-panic-about-anthropics-b4a9e4

    The final-text check failed: citations_correct.

  165. HeldBy this siteGary Marcus

    Held record marcus-no-need-to-panic-about-anthropics-828192

    The final-text check failed: rationale_supported_by_evidence, citations_correct; overclaims: The harness comparison also changes runtime and cost. [e3].

  166. HeldBy this siteGary Marcus

    Held record marcus-no-need-to-panic-about-anthropics-0827a4

    The final-text check failed: citations_correct.

  167. HeldBy this siteGary Marcus

    Held record marcus-no-anthropic-did-not-call-for-a-pause-e2c9d8

    The final-text check failed: citations_correct.

  168. Assessment publishedBy this siteGary Marcus

    Anthropic will keep rushing ahead with AI development and point to the "least cautious actors" to justify it.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-no-anthropic-did-not-call-for-a-pause-21ca94-a1.

  169. HeldBy this siteGary Marcus

    Held record marcus-new-ways-to-corrupt-llms-fa808c

    e1 is dated 2025, which is not inside the period described (… to 2025-12-14); it can only be context

  170. Assessment publishedBy this siteGary Marcus

    LLM vulnerabilities from semantic leakage are likely endless and will never all be patched.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-new-ways-to-corrupt-llms-b7de94-a2.

  171. HeldBy this siteGary Marcus

    Held record marcus-new-ways-to-corrupt-llms-4eedf4

    The anthropic review found the criteria unfair to the speaker's words.

  172. Assessment publishedBy this siteGary Marcus

    Many LLM hallucinations result from overgeneralizing word correlations such as semantic leakage.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-new-ways-to-corrupt-llms-4af24a-a2.

  173. Assessment publishedBy this siteGary Marcus

    DeepSeek shows that vast efficiency improvements in AI are still possible.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-my-conversation-with-chinas-victor-306d47-a1.

  174. Assessment publishedBy this siteGary Marcus

    The finding that geography can be inferred from language corpora is not new; earlier work from 2009 and 2017 showed similar results.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-muddles-about-models-b5604c-a1.

  175. HeldBy this siteGary Marcus

    Held record marcus-muddles-about-models-a9cd0f

    The anthropic review reads the claim differently.

  176. HeldBy this siteGary Marcus

    Held record marcus-muddles-about-models-81f2cc

    The openai review found the criteria unfair to the speaker's words.

  177. HeldBy this siteGary Marcus

    Held record marcus-muddles-about-models-2243c5

    e1 is dated 2023, which is not inside the period described (2023-01-01 to 2023-10-05); it can only be context

  178. Assessment publishedBy this siteGary Marcus

    We can be absolutely sure that the AI task-length 'time horizon' will not keep doubling until time horizons reach 580 times the age of the universe.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-misplaced-panic-over-ai-progress-c1af8f-a1.

  179. HeldBy this siteGary Marcus

    Held record marcus-misplaced-panic-over-ai-progress-aadf56

    hedged language is not scored, even on a plain reading

  180. HeldBy this siteGary Marcus

    The METR time-horizon graph pertains only to software-development tasks, not general intelligence.

    The anthropic review could not run (error).

  181. HeldBy this siteGary Marcus

    Held record marcus-misplaced-panic-over-ai-progress-8ad2a8

    The anthropic review could not run (error).

  182. HeldBy this siteGary Marcus

    Held record marcus-misplaced-panic-over-ai-progress-38d135

    The anthropic review could not run (error).

  183. HeldBy this siteGary Marcus

    AI progress will not keep doubling indefinitely.

    The anthropic review could not run (error).

  184. Assessment publishedBy this siteGary Marcus

    Sam Altman's OpenAI competes directly with Microsoft, even though Satya Nadella offered to rescue Altman after he was fired.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-microsoft-and-openais-increasingly-e654d3-a1.

  185. HeldBy this siteGary Marcus

    Held record marcus-microsoft-and-openais-increasingly-dbe44f

    The final-text check failed: citations_correct.

  186. Assessment publishedBy this siteGary Marcus

    Beyond partnering with Mistral, Microsoft is building a large model in-house that is big enough to compete directly with GPT-4.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-microsoft-and-openais-increasingly-d040d5-a1.

  187. Assessment publishedBy this siteGary Marcus

    Microsoft has not respected OpenAI's stated wish that its software not be used for military purposes.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-microsoft-and-openais-increasingly-b90ba6-a1.

  189. HeldBy this siteGary Marcus

    Held record marcus-microsoft-and-openais-increasingly-750de6

    The openai review found the criteria unfair to the speaker's words.

  190. Assessment publishedBy this siteGary Marcus

    Claiming that foundation model regulation was added late to the EU AI Act text is a major misrepresentation.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-metas-chief-ai-officer-is-lying-about-8864b2-a1.

  191. Assessment publishedBy this siteGary Marcus

    Foundation models (general purpose AI) were first mentioned in the EU AI Act process in mid-May 2022, before ChatGPT's release.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-metas-chief-ai-officer-is-lying-about-760b20-a1.

  192. Assessment publishedBy this siteGary Marcus

    Big tech executives are trying to undermine the newly negotiated EU AI Act compromise after the fact.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-metas-chief-ai-officer-is-lying-about-2a9000-a1.

  193. Assessment publishedBy this siteGary Marcus

    Meta used all of Marcus's works in the database without his permission.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-meta-pirated-at-least-101-of-my-books-eccadd-a1.

  194. Assessment publishedBy this siteGary Marcus

    Meta pirated over 100 of Gary Marcus's books and scientific articles via the Libgen database alone.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-meta-pirated-at-least-101-of-my-books-290ec1-a1.

  195. HeldBy this siteGary Marcus

    Held record marcus-math-is-hard-if-you-are-an-llm-and-d0ce23-2

    The anthropic review found the criteria unfair to the speaker's words.

  196. HeldBy this siteGary Marcus

    Held record marcus-math-is-hard-if-you-are-an-llm-and-9c04f0

    The two reviews differ: anthropic historical_observation/not_applicable/unresolved, openai historical_observation/not_applicable/mixed.

  197. Assessment publishedBy this siteGary Marcus

    The LLM does not induce an algorithm that actually computes multiplication.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-math-is-hard-if-you-are-an-llm-and-577ed6-a1.

  198. Assessment publishedBy this siteGary Marcus

    The MathGLM paper does not prove its titular claim that GPT can solve mathematical problems without a calculator.

    First assessment published after both reviews and the final-text check passed.

    Assessment marcus-math-is-hard-if-you-are-an-llm-and-4f12b3-a2.

  199. HeldBy this siteGary Marcus

    Held record marcus-llms-dont-do-formal-reasoning-and-b0ebef

    e2 is dated 2024-10, which is not inside the period described (2024-01-01 to 2024-10-11); it can only be context

  200. HeldBy this siteGary Marcus

    Held record marcus-llms-dishonest-unpredictable-and-ff4c7a

    The anthropic review found the criteria unfair to the speaker's words.