Claims checked against evidence, page 26 of 31
Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.
Claims checked against evidence (3100)
Each of these claims was checked against dated evidence.
Page 26 of 31: claims 2501 to 2600 of 3100, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: SupportedSaid
“For one thing, in the expert population, the effect is in the predicted direction for all five tasks (ideation, acquisition, magnification, formulation, and release).”
What happened: Joining the complete response file to e1 confirms higher expert threshold-success rates with model access on every task. The date identifies the published dataset being assessed, not an inferred experiment date. (OpenAI, )
Assessment: SupportedSaid
“In fact, if you simply collapsed all the measurements of accuracy, and did the single most obvious test here, a simple t-test, the results would (as Footnote C implies) be significant.”
What happened: Summing within participants and comparing the two assignment groups gives a positive difference with two-sided p = 0.01908. This remains significant while respecting repeated measurements. (OpenAI, )
Assessment: SupportedSaid
“Any fool can simply eyeball these data and see that in virtually every condition, people who have access both to the internet and LLMs (darker bars) are more accurate than people without (lighter bars).”
What happened: Nine task differences are positive. Student acquisition is the sole exception, with a difference of −1.08 points. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: ContradictedSaid
“As a journal reviewer, I would also note that they reported neither the p value they obtained, nor the test of statistical significance they applied; in a competent peer-reviewed journal, that’s a no-no.”
What happened: The methods paragraph names one-sided t-tests and the Mann–Whitney check. The applied tests were therefore identified. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Gary Marcus is standing firmly by his prediction that 2024 will be the year of Generative AI litigation.”
What happened: The review documents a substantial active litigation landscape by midyear. Its count includes earlier filings and must not be presented as 25 new lawsuits in 2024. (Copyright Alliance, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“What unifies all of the above is that current systems are good at local coherence, between words, and between pixels, but not at lining up their outputs with a global comprehension of the world.”
What happened: The authors describe the contemporary contrast between linguistic fluency and explanatory inconsistency. Their experiments concern explanation consistency, not a comprehensive measurement of factual world knowledge. (arXiv; Yanda Chen and colleagues, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“It has become apparent that large language models, OpenAI’s core technology, sometimes leak private data, and that this problem has not been resolved; hallucinations too remain problematic.”
What happened: This is the researchers’ dated assessment of the continuing vulnerability, not a claim that their experiments occurred on publication day. They explain why a word-repetition filter would address one exploit without eliminating memorization and divergence. (Authors of Scalable Extraction of Training Data from (Production) Language Models, )
Assessment: SupportedSaid
“And candor of the company as a whole will remain an issue.”
What happened: Vox compared leadership claims of ignorance about equity-related exit provisions with signed company documents. This establishes a substantive corporate candor dispute, without conclusively proving what each executive knew. (Vox, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“OpenAI’s unusual relationship with Microsoft is sure to receive considerable scrutiny.”
What happened: The decision describes examination of internal documents, information-request responses and the partnership’s commercial operation. Its analysis covers governance, compute supply and intellectual-property rights, including developments after the forecast. (UK Competition and Markets Authority, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“We can of course expect related problems of discomprehension in efforts to use these systems to address copyright issues, as I wrote earlier this week.”
What happened: The public report documents a completed evaluation using generative vision-language models for copyright detection. The event date is the release of that empirical report; individual testing dates are not specified. (arXiv; Xu and colleagues, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“More than that, GenAI developers won’t get off so easy, and lawyers looking at copyright infringement will still have plenty of work to do.”
What happened: The amended-complaint ruling kept Copyright Act claims alive against the image-generation defendants after an earlier dismissal round. It establishes continued contested litigation, not ultimate infringement liability. (U.S. District Court, Northern District of California, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“The opposite problem happens too; guardrails can wind up too permissive. This is still happening over a year into the chatbot revolution.”
What happened: The investigator reports continuing toxicity failures in other LLMs as of this update. The same account says Google had improved PaLM 2's safeguards, so it does not establish that the original PaLM 2 failures remained reproducible. (Montreal AI Ethics Institute; research summary by Ashique KhudaBukhsh, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“And you know what, OpenAI knows this argument is bullshit; they know that licensing is a possibility, because they have been running around working on licensing deals.”
What happened: AP directly confirms an actual licensing arrangement before Marcus’s post. Entering the agreement demonstrates institutional awareness of licensing as an available option. (The Associated Press, )
Assessment: SupportedSaid
“To begin with, the threat is empty: open source LLMs are already out in the wild, and a large number of people know how to build them.”
What happened: The announcement establishes a pre-post public model release and explicitly provides local-use instructions. This supports technical availability independent of OpenAI’s service. (Mistral AI, )
Assessment: SupportedSaid
“And what Southen and I showed is that generative image programs are prone to exactly the same kind of regurgitation that is at issue.”
What happened: The experiment establishes generation of memorized training images. This directly supports the technical meaning of regurgitation, without establishing infringement or a typical-use frequency. (arXiv; Nicholas Carlini and colleagues, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“OpenAI, despite its name, has not been transparent about what it has been trained on.”
What happened: The system card describes broad source categories and filtering methods but does not identify the underlying training corpus or its substantive source inventory. This supports limited transparency, not an assertion that OpenAI disclosed nothing. (OpenAI; reproduced at dalle-3.pages.dev, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“A good system should give the user a manifest of sources; current systems don’t.”
What happened: The authors describe the state at their December 2023 writing date and identify the missing provenance relationships needed for attribution. This supports the contextual reading about underlying source manifests, although it is a research synthesis rather than an exhaustive product audit. (An MIT Exploration of Generative AI; author-hosted copy, )
Assessment: SupportedSaid
“Generative AI systems like DALL-E and ChatGPT have been trained on copyrighted materials;”
What happened: The announcement confirms that licensed contributor imagery had already been used to train DALL-E by this date. Licensed copyrighted material satisfies the claim regardless of whether its use was authorized. (Shutterstock, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“It’s also hardly the first time AI is helped with mathematics.”
What happened: This firsthand report documents AI contributing to mathematical research two years before FunSearch. That is sufficient to establish the claimed predecessor. (DeepMind, )
Assessment: SupportedSaid
“There is no evidence that the problem was heretofore “unsolvable””
What happened: This predating result demonstrates constructive progress on the same lower-bound problem. It supports treating the task as an open optimization question rather than an established impossibility. (arXiv / Fred Tyrrell, )
Assessment: SupportedSaid
“Human mathematicians had to write problem-specific code for each separate mathematical problem.”
What happened: The specification section requires a coded evaluator and an initial program. Figure 2 supplies different evaluation and surrounding code for cap sets and bin packing, supporting the reading that humans supplied task-specific code. (Nature / Google DeepMind author manuscript, )
Assessment: UnresolvedSaid
Assessment: ContradictedSaid
“It came in the middle of May 2022.”
What happened: The November 2021 compromise text explicitly mentions general-purpose AI and contains a dedicated Article 52a. This predates the claimed first mention in May 2022. (Council of the European Union, Slovenian Presidency, )
Assessment: SupportedSaid
“The phrase “Added late in the text” — which tries to paint the regulation of foundation models as some kind of sneaky last minute move — is a massive, massive misrepresentation.”
What happened: The May 2022 proposal contains general-purpose AI duties in Article 4b and expressly exempts smaller enterprises from them. Proposed regulation therefore existed well before December 2023, although this does not date every provision in the eventual agreement. (Council of the European Union, French Presidency; copy hosted by the Future of Life Institute, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“ELIZA (built in 1965-1966) beat GPT 3.5.”
What happened: The same results paragraph reports ELIZA at 27%, above both GPT-3.5 configurations. These are overall judged-human rates in the experiment Marcus cited. (arXiv; Cameron Jones and Benjamin Bergen, )
Assessment: SupportedSaid
“And, sorry Sam, when you actually look at the data. humans are still ahead.”
What happened: The section reports a 63% human success rate and 41% for the strongest GPT-4 prompt. All other tested AI configurations scored lower. (arXiv; Cameron Jones and Benjamin Bergen, )
Assessment: SupportedSaid
“The whole project was quietly dropped a year or two later.”
What happened: The next sentence dates the move to the preceding October, and the reproduced company statement says further robotics research stopped. This establishes October 2020 rather than treating the July 2021 reporting date as the shutdown date. (TechCrunch, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Note that Gates and Altman have both been dropping hints, and GPT-5 isn’t here after a year despite immense commercial desire.”
What happened: The sentence reports Gates predicting initial stagnation. The surrounding account also anticipates later improvements, so his skepticism was qualified. (Handelsblatt, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“And that’s acknowledging the fact that LLMs — even those developed a legion of well-resourced and well-fed PHDs, like GPT-4, and based on a well-curated dataset of high-quality material — have a habit of hallucinating facts, figures, and events.”
What happened: OpenAI acknowledges factual hallucinations in GPT-4. Its accompanying report of improvement over earlier models does not remove that recurring limitation. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Its pioneering, world-leading AI Act is on the ropes.”
What happened: The article reports a negotiating walkout and interviews Kai Zenner, who put timely passage at a 50–50 chance after the impasse. It also records unresolved disagreements described by co-rapporteur Tudorache, supporting a serious threat to agreement. (IAPP, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“And I do mean “we”; what the EU decides will affect all of us.”
What happened: In this section Anthropic expressly attributes watermarking to EU AI Act compliance and states that it applies the policy globally. This establishes a concrete worldwide development-policy consequence, while leaving rollout timing and future regional approaches open. (Anthropic, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“getting their algorithm to work for one object (which, cheesily, turned out to be a special Cube instrumented with sensors and LEDS, unlike a Rubik’s cube you’d buy in a store) in carefully controlled lab circumstances hardly guaranteed that the solution would work more broadly in the complex open-ended real world.”
What happened: The preceding discussion identifies arbitrary-object interaction as a long-term goal and a completely standard cube as unfinished work. Sections 3 and 7 document the laboratory apparatus and sensor-equipped Giiker experiments, rather than an established solution for open-ended manipulation. (OpenAI; original report hosted by Matthias Plappert, )
Assessment: SupportedSaid
“AV’s still aren’t general enough that you can just plop down a car that was tuned in pilot studies in Menlo Park, SF and Arizona and expect it to drive gracefully and safely in Sicily or Mumbai.”
What happened: Waymo describes initial manual driving, subsequent specialist feedback, virtual evaluation of the new location, and refinements from local experience. This directly supports the need for preparation before expecting dependable operation in unfamiliar places. (Waymo, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“It’s fun to compare the two—they have so many differences—even though neither can be trusted.”
What happened: The preserved table reports hallucinations in 3.0% of GPT-4 summaries and 3.5% of GPT-3.5 Turbo summaries, despite providing source documents. These recurring errors support independent checking; the percentages apply to this summarization test, not all GPT answers. (Vectara, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“OpenAI’s strength has been in scaling up an existing paradigm (Transformers) that was developed elsewhere (Google), not developing new paradigms.”
What happened: The original paper introduces the Transformer and lists Google Brain and Google Research affiliations, alongside a University of Toronto affiliation. This supports the Google research provenance without implying that every author had only a Google affiliation. (Ashish Vaswani and colleagues / arXiv, )
Assessment: SupportedSaid
“And the overriding nonprofit mission was always going to be at odds with a pure for profit play.”
What happened: The section gives the nonprofit board control and says investors and employees accept the Charter's priority even at the expense of their financial stakes. Those explicit constraints support the structural reading of conflict with a purely profit-driven investment. (OpenAI, )
Assessment: SupportedSaid
“There has never been a solution to the hallucination problem, and it’s not clear that there will be until we find a new paradigm, as Altman himself partly acknowledged last week, when he did his best Gary Marcus impression.”
What happened: OpenAI acknowledged residual factual hallucinations in GPT-4. The surrounding discussion reports improvements, not elimination. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“and then, perhaps in keeping with my surmise that Sam’s bluff was called, negotiations broke down altogether.”
What happened: The report directly supports the failure of the negotiations at the Sunday-night leadership transition. It does not establish the proposed bluff explanation. (TIME, )
Assessment: SupportedSaid
“Artificial intelligence has become one of the few reliable roads to capital in the current economy, with venture capitalists sinking more than a quarter of their startup dollars into AI-related companies in 2023.”
What happened: Crunchbase directly reports a share above 25% for American startups. Its accompanying analysis reports increasing AI investment amid a broad funding contraction, supporting the relative-resilience reading. (Crunchbase News, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Altman will be fine. OpenAI, however, is an entirely different question.”
What happened: OpenAI officially confirmed Altman's return to the same chief executive role. This resolves the career setback under the stated ordinary reading. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“We pretty obviously don’t have a say.”
What happened: The article describes exclusive director election and removal rights and board control over changes to those rules. This supports the inference that the public lacked a formal role; contemporaneous reporting is used because the available primary bylaws are an older filing. (WIRED, )
Assessment: MixedSaid
“The spirit of the original arrangement was that everything that the for-profit did was supposed to be in the service of the non-profit. A board with no financial interest was supposed to look out for humanity.”
What held: The announcement describes the structure's humanitarian purpose. Its background paragraph expressly says the 2019 restructuring preserved the nonprofit's mission, governance, and oversight. (OpenAI, )
What did not: The original rules permitted some directors to hold financial stakes, contradicting an entirely disinterested-board requirement. The following sentence reserved decisions involving conflicts between investors and the nonprofit mission to directors without stakes. (OpenAI, )
Assessment: SupportedSaid
“All signs are that those financially-interested stakeholders will quickly emerge victorious. (Arguably, they already have).”
What happened: The announced settlement provided both Altman's return and a replacement board two days after the post. November 22 is used conservatively for availability because the search record and page display differ across the overnight publication boundary. (Reuters, republished by Yahoo Finance, )
Assessment: SupportedSaid
“In reality, things have gone quite differently. When the nonprofit Board felt it was prudent (for undisclosed reasons) to remove Altman, per the goals of the nonprofit that they were trying to represent, at the risk of potentially devastating commercial potential, those with a stake in the nominally subordinate for-profit (both employees and investors) quickly set to work to push out the board and to undo its decisions.”
What happened: The report describes Murati negotiating Altman's return and investors led by Microsoft and Thrive demanding that directors step down. It establishes coordinated executive and investor opposition during the period claimed. (Bloomberg, republished by Yahoo Finance, )
Assessment: SupportedSaid
“The fact that the governance of one of the most visible AI companies in the world can change literally overnight”
What happened: The dated announcement establishes the CEO transition. Its body makes Murati's appointment effective immediately and also announces Altman's departure from the board and Brockman's departure as chair, establishing a same-day change in governing personnel. (OpenAI, )