Claims checked against evidence, page 20 of 31
Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.
Claims checked against evidence (3100)
Each of these claims was checked against dated evidence.
Page 20 of 31: claims 1901 to 2000 of 3100, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“LLMs are already dangerous, despite their limits.”
What happened: The researchers demonstrated unfavorable job assignments and harsher hypothetical criminal judgments associated with African American English. This establishes a concrete discriminatory risk, while the experiment does not establish that employers or courts actually imposed these outcomes. (arXiv, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: ContradictedSaid
“The products aren't coming.”
What happened: The disclosure dates general availability to May 2025 and reports continued growth in weekly users and business subscriptions in February 2026. Its reported commercial scale supplies evidence that adoption extended beyond an experimental launch. (Anthropic, )
Assessment: ContradictedSaid
“I also want to be clear that none of these companies ever had a plan.”
What happened: Microsoft specified concrete generative AI functions inside Word and other work applications. That is a product and deployment strategy beyond purchasing GPUs, regardless of whether the promised benefits were achieved. (Microsoft, )
Assessment: SupportedSaid
“OpenAI currently spends $2.35 to make $1, and Microsoft CFO Amy Hood said that OpenAI would cut into Microsoft's profits this October, losing it a remarkable $1.5 billion, "mainly because of an expected loss from OpenAI" according to CNBC.”
What happened: These estimates imply $8.7 billion of expenses and a ratio rounding to $2.35. CNBC also reports exclusions, including equity-based compensation, so this supports the cited estimate rather than comprehensive accounting expenses. (CNBC, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: ContradictedSaid
“The entire tech industry has become oriented around a dead-end technology that requires burning billions of dollars to provide inessential products that cost them more money to serve than anybody would ever pay.”
What happened: The preceding sentence identifies 2024 revenue of $4 billion and inference compute and hosting costs of $2.4 billion. This reported positive serving margin supplies a counterexample to the categorical assertion, without establishing company-wide profitability. (Epoch AI, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: ContradictedSaid
“Generative AI's products have effectively been trapped in amber for over a year.”
What happened: Developers could use the API to have Claude inspect screens, move a cursor, click and type. This materially extends possible workflows, although Anthropic describes the release as experimental and error-prone. (Anthropic, )
Assessment: UnresolvedSaid
Assessment: MixedSaid
“The hallucination problem is one that is nowhere closer to being solved — and, at least with the current technology — may never go away, and it makes it a non-starter for a great many business tasks, where you need a high level of reliability.”
What held: Testing on May 23–27 found frequent hallucinations in a specialized legal research product. This supports the barrier to unverified, high-reliability legal work under the stated interpretation, but does not establish the number of business tasks excluded. (arXiv, )
What did not: Table 2 reports SimpleQA hallucination rates of 0.61 for GPT-4o and 0.44 for o1-preview, with improvements on the other listed tests. The same paragraph reports contrary anecdotal feedback and cautions against generalizing beyond these evaluations. (OpenAI, )
Assessment: ContradictedSaid
“The models aren't going to get better.”
What happened: The report identifies MAI-Thinking-1's base-model training on 8K GB200 GPUs and reports 84.2% GPQA Diamond pass@1, exceeding the pre-post results in e3. It uses the same 198-question Diamond subset; its large output budget and grading procedure limit exact comparability. (Microsoft AI, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Two years later Google’s search share hasn’t diminished all that much.”
What happened: This supplies the launch-era worldwide baseline. Together with e2, it supports a modest decline; historical primary figures were not retrievable through the page reader. (Local Dialog, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“GPT-4 was better than GPT-3 in almost every way; o1 is a far less uniform improvement.”
What happened: The study evaluates legacy GPT-3 models and GPT-4 under aligned settings across more than twenty benchmarks and seven capability categories. Its overall analysis reports broad improvements across those categories from GPT-3 to GPT-4. (Zheng et al., ByteDance and University of Illinois Urbana-Champaign, arXiv, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“(In some cases o1 doesn’t even do better than GPT-4).”
What happened: The stated count implies that o1 did not outperform GPT-4o in the remaining three subcategories. It supports the limited 'some cases' claim under the contextual reading of GPT-4. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Many other Tech CEOs and others, including Marc Andreeseen (A16Z), Alexandr Wang (ScaleAI), have also used the phrase “hitting a wall” and the idea of diminishing returns in the last couple weeks.”
What happened: Wang publicly used the 'wall' framing on Nov 20, 2024, three days before the post. He partly agreed that a wall had been hit, and the report links it to diminishing returns from training. (The Verge, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Scaling was always a hypothesis.”
What happened: Early scaling research built its claims from empirical measurements across several tasks. This supports the distinction between observed regularities and guarantees about their indefinite continuation. (arXiv; Hestness et al., )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“They are merely empirical generalizations that held for a certain period time, when there was enough fresh data and compute.”
What happened: The authors explicitly characterize their loss relationships as empirical. Their reported measurements cover finite ranges of model size, data and training compute. (arXiv; Kaplan et al., )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Many of us, including Melanie Mitchell, Subbarao Kambahapati, Emily Bender, Ernie Davis, etc. have been emphasizing for ages that there are principled limits with LLMs.”
What happened: Bender and Koller's paper makes an explicit conceptual-limit argument more than four years before the post. The metadata gives July 2020; July 31 is used as a conservative availability bound. (Association for Computational Linguistics, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“People aren’t ignoring GenAI; they are waiting to see if it will work.”
What happened: A third of the firms were still testing. That fits 'waiting to see if it will work.' (CNBC (syndicated by NBC San Diego), )
Assessment: SupportedSaid
“But only 25% of the correspondents thought it was worth it.”
What happened: Matches his 25% yes, 25% no, 50% too-soon breakdown exactly. (CNBC (syndicated by NBC San Diego), )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“(is OpenAI’s o1 100 times better than GPT4? Certainly not by their own data)”
What happened: On AIME, one of OpenAI's best showcases for o1, the gain is about sixfold (74% vs 12%), far below 100x. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“It’s just fantasy to imagine selling 200 times as many humanoid robots in the nearish term when nobody knows how to build a single safe, reliable, generally useful humanoid right now, at any price.””
What happened: The accompanying expert discussion says simple tasks still require substantial training and performance remains strongest in controlled, repetitive settings. This supports the ordinary reading that dependable general usefulness had not been established. (IEEE Transmitter, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: ContradictedSaid
“The difference between GPT-3 and GPT-4 was by all accounts massive; the difference between GPT-4 and o1 is by all accounts modest.”
What happened: The company reported substantial preference gains across three important reasoning domains for the actual preview model. This challenges the asserted agreement about only modest improvement, while remaining a company-run evaluation. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Jailbreaks aren’t new, but even after years of them, the tech industry has nothing like a robust response.”
What happened: The study evaluates four defenses and reports substantial vulnerability to human multi-turn attacks despite strong results against automated single-turn attacks. This supports the ordinary reading that established defenses lacked broad robustness. (arXiv; Scale AI and UC Berkeley researchers, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“It isn’t. You saw a prototype and heard a speech.”
What happened: Explicitly identifies the unveiled Cybercab as a prototype. The article's headline uses arrival language, but its description supports Zitron's distinction. (The Verge, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: MixedSaid
“far too many simply ran exactly what Musk said and acted as if he’d proven that these were actual robots walking around and doing things on their own accord, as opposed to what was blatantly obvious: that they were controlled remotely by human operators.”
What held: Reports a bartender robot's acknowledgment of assistance, alongside accounts of teleoperated guest interactions. (Ars Technica, )
What did not: Contradicts including walking in a blanket account of remote control. The reported operation combined autonomous locomotion with human-assisted interactions. (Ars Technica, )
Assessment: SupportedSaid
“Fall off on increasingly large multiplication problems has repeatedly been observed, both in older models and newer models.”
What happened: For older models (GPT-3, ChatGPT, GPT-4), multi-digit multiplication accuracy falls from near perfect to zero as problem size grows. (arXiv (Dziri et al., NeurIPS 2023), )
Assessment: MixedSaid
“I strongly believe the current results are robust.”
What held: An independent team built a controlled benchmark with irrelevant context (IC) and found LLMs significantly sensitive to it. This reproduces the core GSM-NoOp-style finding. (arXiv (Yang et al.), )
What did not: A peer-reviewed re-analysis found the GSM-Symbolic drop significant for only 8 of 20 models, and partly explained by larger numbers in the new dataset. That cuts against the robustness of that headline result. (arXiv (Długosz, Oliveira, Díaz-Rodríguez; accepted EMNLP 2026), )
Assessment: SupportedSaid
“Performance is ok on small problems, but quickly tails off.”
What happened: o1-preview scored 97.8% on small problems but 23.63% on 6-20 block problems, and its correct answers on the larger set were mostly the shorter plans. That is good performance on small problems with a quick fall-off. (arXiv (Valmeekam, Stechly, Kambhampati), )
Assessment: SupportedSaid
“Failure to follow the rules of chess is another continuing failure of formal reasoning:”
What happened: On a 1,000-puzzle benchmark in mid-2024, the strongest model tested (GPT-4o) made illegal moves 12.7% of the time, and most other LLMs made more illegal moves than legal ones. That matches a continuing failure to follow chess rules before the post. (The Register, )
Assessment: UnresolvedSaid
Assessment: ContradictedSaid
“We have been told that AI will “solve physics” (per Sam Altman), make major breakthroughs in medicine, battery technology, etc, but so far none of that has materialized.”
What happened: The paper reports demonstrated accuracy in the CASP14 assessment conducted in May–July 2020. Together with e2, it establishes an achieved major research advance rather than an untested promise. (Nature, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: ContradictedSaid
“That product is extremely commoditized. The free version of ChatGPT is effectively identical to the free version of Anthropic's Claude, Meta's AI assistant, Microsoft's Copilot, and even Twitter's "Grok."”
What happened: The launch announcement places access in premium subscription tiers during the target quarter. The explicit premium-only history in e2 corroborates that this was not an ordinary free consumer offering. (xAI, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“All of this is happening at a time when an astronomical amount of talent is leaving the company — the talent that OpenAI desperately needs to build products that people will actually pay for.”
What happened: The simultaneous departure announcements cover the chief technology officer, chief research officer and a research vice president. Those roles support the ordinary reading of a substantial concentration of relevant leadership departures. (The Register, )
Assessment: ContradictedSaid
“To hit a revenue target of $11.6 billion in 2025, OpenAI would need to increase revenue from ChatGPT customers by 310%.”
What happened: The derived counterexample reaches the target with a 213.5% increase in the stipulated ChatGPT baseline, assuming the stated mix and a full year at the resulting rate. This is a calculation from reported premises, not a measured revenue outcome. (CNBC, )
Assessment: SupportedSaid
“To be abundantly clear, as it stands, OpenAI currently spends $2.35 to make $1.”
What happened: The reported estimates imply $8.7 billion of expenses on their loss basis and an expense-to-revenue ratio of 2.35135. This supports the arithmetic reading, not a claim about realized expenditure. (CNBC, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“OpenAI's products are increasingly-commoditized, with Google, Meta, Amazon and even Microsoft building generative AI models to compete.”
What happened: The retrospective comparison identifies capability convergence by Llama 3.1, released during the target quarter. It supports increasing model substitutability while also documenting historical lags behind closed models. (Epoch AI, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid