Claims checked against evidence by Gary Marcus, page 16 of 19
Deadlines are judged as of 2026-10-29. About Gary Marcus and the sources covered.
Claims checked against evidence by Gary Marcus (1870)
Each of these claims was checked against dated evidence.
Page 16 of 19: claims 1501 to 1600 of 1870, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“It highlights the inadequacy of the voluntary agreements such as the one that 7 tech companies recently made with The White House. Those agreements called for watermarking, but because they are voluntary, they do nothing to prevent bad actors from rolling their own automated misinformation software.”
What happened: The full document assigns undertakings to participating companies and contains no obligation on independent nonparticipants. It also includes safeguards against misuse and unauthorized access, which could provide indirect benefits without binding outside developers. (The White House, publishing participating companies' commitments, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Social media companies better watch out, too; they will be polluted in much the same way as the internet as a whole.”
What happened: The report documents multiple covert campaigns using generated material on Telegram, X, Facebook, Instagram, and websites, including Doppelganger and Zero Zeno. These establish repeated distribution through the mechanism described, while the report also says AI had not meaningfully increased audience reach. (OpenAI, )
Assessment: SupportedSaid
“And it’s on: the LLM-driven enshittification of the internet.”
What happened: NewsGuard identified 49 apparently AI-generated sites using published chatbot artifacts and content inspection, including a fabricated report of President Biden's death. Its report also says most identified sites did not advance misinformation, and some operators disputed its characterization. (NewsGuard, )
Assessment: SupportedSaid
“And of course LLMs are fully capable of make up truly terrible advice, too, like this disconcerting recipe from last week:”
What happened: The report dates Hehir's dangerous recipe to August 4 and identifies its LLM. It specifies chloramines and reports that attempted reproduction on August 10 was blocked. (Ars Technica, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“The underlying technology—often referred to as Foundation Models, because systems are fine-tuned on top of large pretrained (foundation) model—is deeply unstable and thus difficult to engineer into reliable products.”
What happened: Comparisons of March and June model versions document formatting failures that prevented code execution and caused a LangChain parser to fail, directly connecting model behavior to integration difficulty. The results also limit the claim: GPT-4's agent performance improved, and removing non-code wrappers increased its code-test success from 52% to 70%, so behavioral change did not imply declining underlying capability. (arXiv; Lingjiao Chen, Matei Zaharia, and James Zou, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“We have not one, but many, serious, unsolved problems at the core of generative AI — ranging from their tendency to confabulate (hallucinate) false information, to their inability to reliably interface with external tools like Wolfram Alpha, to the instability from month to month (which makes them poor candidates for engineering use in larger systems).”
What happened: OpenAI explicitly acknowledged continuing factual errors in its released model. The same section reports improvements, so persistence should not be confused with lack of progress. (OpenAI, )
Assessment: SupportedSaid
““Scaling” systems by making them larger has helped in some ways, but not others; we still really cannot guarantee that any given system will be honest, harmless, or helpful, rather than sycophantic, dishonest, toxic or biased.”
What happened: The study documents tasks where increasing scale worsened performance despite broader gains in prediction and other tasks. It also reports exceptions and trend reversals, so it does not establish that scaling generally fails. (Ian R. McKenzie and colleagues, arXiv, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“The technology we have now is built on autocompletion, not factuality.”
What happened: The report explicitly identifies predictive completion as GPT-4’s base objective. It separately reports factuality gains from post-training, supporting the foundation reading while qualifying a literal denial of factuality work. (OpenAI, arXiv, )
Assessment: UnresolvedSaid
Assessment: MixedSaid
“In just a few short months, the race to regulate AI, possibly at an international level, has gone from barely on the table to heated.”
What held: This primary government announcement describes summit planning, recent G7 discussions and meetings with frontier AI companies. It supports the high-level activity component by June, but does not establish a marginal starting point. (UK Prime Minister's Office, )
What did not: The February meeting was already the committee's fourth plenary, with negotiations and draft convention provisions under examination. That challenges the description of international AI regulation as barely on the table early in 2023. (Council of Europe, Committee on Artificial Intelligence, )
Assessment: SupportedSaid
“We also know, for example, that Bing has defamed people, and it has misread articles as saying the opposite of what they actually say, in service of doing so.”
What happened: The same post identifies Turley's USA Today rebuttal among Bing's sources for the false allegation. It directly connects the reputational falsehood with the reversed source meaning. (Will Oremus, reporting on Twitter, )
Assessment: SupportedSaid
“Bing relies on GPT-4, and we (eg, in the scientific community) don’t have access to how GPT-4 works, and we don’t have access to what data its trained on (vital, since we know that systems can bias eg political thought and hiring decisions based on those undisclosed data) — that’s about as far away from transparency as we could be.”
What happened: The contemporaneous complaint identifies the specific missing disclosures. Its statement is corroborated by OpenAI's own report; the complaint itself is not an adjudicated regulatory finding. (Center for AI and Digital Policy, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“But their current, actual products are definitely not transparent, and at least in some ways, demonstrably not safe.”
What happened: The researchers describe Bing as a system they could test without access to its internals. They identify GPT-4 as its underlying model and distinguish available high-level descriptions from unavailable implementation details. (Kai Greshake and colleagues / arXiv, )
Assessment: SupportedSaid
“Meanwhile, ChatGPT plugins (produced by OpenAI, which they have a close tie with) open a wide range of security problems: those plugins can access the internet, read and write files, and impersonate people (e.g., to phish for credentials), all alarms to any security professional.”
What happened: The launch documentation establishes browsing and file-processing capabilities and reports fraudulent-email and prompt-injection findings without safeguards. It also describes sandboxing, network controls, and a GET-only browser, which limit the claim's scope. (OpenAI, )
Assessment: SupportedSaid
“Truthfulness is one of the major gaps in current systems, as evidenced by hallucinations, weird defamatory accusations etc.”
What happened: OpenAI explicitly identifies factual unreliability as a continuing limitation of its newly released leading model. The surrounding guidance calls for care and human review in consequential uses. (OpenAI, )
Assessment: SupportedSaid
“Getting GPT-5 first will therefore not directly bring China or US to AGI first, per Premise 2.”
What happened: The announcement establishes actual possession and release of GPT-5, satisfying the acquisition condition under the stated reading. It does not establish exclusive national access. (OpenAI, )
Assessment: SupportedSaid
“Regulation is NOT always bad for technology, e.g., regulation around the environmental impact of cars has spurred advances in electric cars; rising fuel standards have had a positive impact as well.”
What happened: The study uses automotive patent data for 2000–2016 and estimates a positive effect of fuel-economy and greenhouse-gas standards on clean invention, driven by electric and hydrogen vehicles. The event year marks the end of the historical sample, not the publication date. (CESifo, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“As most experts realize, the reality is that current AI doesn’t “decide to teach itself”, or even have consistent beliefs.”
What happened: The report describes externally organized training and explicitly states in its limitations that GPT-4 does not learn from its experience. This supports the operational reading of the self-teaching clause. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: ContradictedSaid
“Essentially none of the fancy cognitive things GPT-4 is rumored to do stand up.”
What happened: The study tested new cognitive-reflection and Linda/Bill problems, finding strong GPT-4 performance across repeated trials. Figure 5 explicitly dates these experiments to March 2023, so the later publication bears on the contemporary model. (Communications Psychology / Europe PMC, )
Assessment: SupportedSaid
“Maarten Sap from Yejin Choi’s lab dove deeper, with a more thorough investigation, and confirmed what we suspected: the claim that large language models have mastered theory of mind is a myth:”
What happened: The extension tests contemporary models, including gpt-4-0314, and finds substantial errors on mental-state questions. These results substantiate the attribution to Sap and are incompatible with robust mastery under the stated reading. (Maarten Sap, Ronan LeBras, Daniel Fried and Yejin Choi / arXiv, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“It’s not smart enough to filter out falsehood, but just smart enough to be dangerous, creating and spreading falsehoods it fails to verify.”
What happened: OpenAI acknowledges factual unreliability in the released system. The same disclosure reports improved factuality and truth discrimination, which limits any literal reading that GPT cannot recognize falsehood at all. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“The fundamental issue is that chatbots are neither reliable nor truthful, and at the same time don’t really grasp things like human psychology.)”
What happened: The developer acknowledges factual and reasoning failures and recommends safeguards for high-stakes use. This supports the practical reliability concern, while the same disclosure reports improvements over GPT-3.5. (OpenAI, )
Assessment: SupportedSaid
“Two reasons, first, because I believe that more incidents like this will happen, perhaps many more, hundreds or thousands, given how rapidly chatbots are being deployed, and the near total lack of regulation on how they can be used.”
What happened: The defendant's motion recounts sustained Character.AI use, a final February 2024 conversation, and the ensuing suicide. It disputes the interpretation that the chatbot encouraged suicide, so the record supports association rather than established causation. (Character Technologies, Inc.; U.S. District Court, Middle District of Florida, )
Assessment: MixedSaid
“Two reasons, first, because I believe that more incidents like this will happen, perhaps many more, hundreds or thousands, given how rapidly chatbots are being deployed, and the near total lack of regulation on how they can be used.”
What held: Google opened public access to another major chatbot during the same period as Bing's rollout. Together with e2, this supports rapid deployment. (Google, )
What did not: The authority imposed an immediate temporary restriction on processing Italian users' data and identified privacy and child-protection deficiencies. This demonstrates enforceable constraints on chatbot operation, although it does not establish comprehensive mental-health safeguards. (Italian Data Protection Authority, )
Assessment: SupportedSaid
“The Chatbot involved was GPT-J, not GPT-3 or ChatGPT (contra some news headlines that were erroneous).”
What happened: The report connects the Belgian incident to Chai's Eliza and identifies Chai's model as GPT-J-derived. The founders describe additional fine-tuning, so this supports the model-family identification rather than an assertion that unmodified GPT-J was deployed. (VICE / Motherboard, )
Assessment: SupportedSaid
“Whether or not you want a pause, the reality is that large language models remain exactly what I have always described them to be: bulls in a china shop, powerful, reckless, and difficult to control.”
What happened: The report documents strong professional and academic benchmark results, supporting substantial capability. Its alignment improvements are also counterevidence to any reading that denies progress in control. (OpenAI via arXiv, )
Assessment: SupportedSaid
“At some level the petition worked; Capitol Hill, for example, pretty clearly got the message that we can’t sit around forever on these issues.”
What happened: Mast responds favorably. (Fox News, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“it didn't call for a ban on the vast majority of AI research, only for a brief pause on one very specific project with a technology that has *known* risks with no known solutions.”
What happened: The requested pause targets training above a capability threshold rather than AI research generally. This document is used only to establish what the letter requested, not to validate its risk assertions. (Future of Life Institute, )
Assessment: ContradictedSaid
“None of the top labs actually is remotely transparent, governance basically doesn’t exist yet, and there are basically no safeguards actually in place.”
What happened: Meta disclosed named data sources and their proportions, preprocessing, architecture, training settings, hardware, and benchmark results, including bias and toxicity assessments. These disclosures provide a substantive counterexample to the assertion that no leading laboratory offered meaningful transparency. (Meta AI / arXiv, )
Assessment: SupportedSaid
“I personally haven't changed; I still think that LLMs are unreliable, and still think that they are a very poor basis for factuality.”
What happened: OpenAI identifies factual and reasoning failures in its released model and recommends care in using its outputs. The same section reports a 40% improvement over GPT-3.5 on internal adversarial factuality evaluations, which qualifies the criticism without establishing dependable absolute reliability. (OpenAI, )
Assessment: SupportedSaid
“A company called Adept.AI just raised $350 million dollars to do just that, to allow large language models to access, well, pretty much everything (aiming to “supercharge your capabilities on any software tool or API in the world” with LLMs, despite their clear tendencies towards hallucination and unreliability).”
What happened: The release confirms the announced $350 million financing. It also describes the long-term goal of training an AI teammate to use every software tool and API, and says funding would support products, model training, and hiring. (Adept, reproduced at adept.futurefabric.co, )
Assessment: SupportedSaid
“But here’s the thing: although a lot of the literature equates artificial intelligence risk with the risk of superintelligence or artificial general intelligence, you don’t have to be superintelligent to create serious problems.”
What happened: The investigation describes a fatal collision while Uber's developmental driving system controlled the vehicle; it failed to classify the pedestrian correctly or predict her path, and its design suppressed emergency braking. This supports a material contribution from limited AI, while the report identifies the distracted operator as the probable cause and also identifies organizational and oversight failures. (National Transportation Safety Board, )
Assessment: MixedSaid
“Microsoft and OpenAI are producing and deploying products with potentially enormous risks at mass scale, and making extravagant, unreviewed and unreviewable claims about them, without sketching serious solutions to any of the potential problems they themselves have identified.”
What held: Microsoft reported millions of active preview users and more than 45 million cumulative chats. This supports substantial public deployment; the separate total for all Bing users is not treated as AI-chat usage. (Microsoft Bing, )
What did not: The report describes reinforcement learning, safety-specific training, expert testing, and abuse monitoring as responses to acknowledged problems. Their publication contradicts the absence of substantive proposals, while the report also acknowledges remaining failures. (OpenAI, )
Assessment: SupportedSaid
“The problem of hallucinations is not solved; reliability is not solved; planning on complex tasks is (as the authors themselves acknowledge) not solved.”
What happened: OpenAI acknowledged that hallucinations remained at launch. Persisting fabricated answers support the related claim that factual reliability was not solved. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“True LLMs don’t (just) memorize, and true that their models make for lousy databases (for a database a hallucination is an outright fail)”
What happened: OpenAI acknowledged factual fabrication in the released model and recommended checks appropriate to the use case. The same section reports improved factuality over GPT-3.5, but explicitly says the problem remains. (OpenAI, )
Assessment: SupportedSaid
“We don’t know how big it is; we don’t know what the architecture is, we don’t know how much energy was used; we don’t how many processors were used; we don’t know what it was trained on etc.”
What happened: Sutskever directly confirms withholding size and training data. The article also documents the absent energy and hardware information and reproduces OpenAI's statement withholding architectural and training details. (The Verge, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“It can see (though we don’t have a lot of details on that yet); it does astonishingly well on a whole bunch of standardized tests, like LSATs, GREs, and SATs.”
What happened: The report documents launch-model image processing and reports approximately the 88th percentile on LSAT, 93rd on SAT reading/writing, 89th on SAT mathematics, 80th on GRE quantitative and 99th on GRE verbal. GRE writing was approximately the 54th percentile; event_date denotes the launch model's capability, not an asserted examination date. (OpenAI, published on arXiv, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Outliers remain a problem, too.”
What happened: OpenAI acknowledges that specially constructed prompts can still defeat GPT-4's safeguards. This supports the ordinary reading that unusual inputs remain a source of failures. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“To me the only solution to the long-term risk issue is to build machines with consensus human values, but we are a long way from knowing how to do that”
What happened: Anthropic explicitly describes robust alignment of powerful systems as unsolved. Its discussion of supervision, generalization and alignment evaluation supports the reading that foundational work remained. (Anthropic, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Still, even though Chomsky’s argument surely could have used considerably more nuance (see my article with Ernest Davis on how not to evaluate GPT), his overall point is correct: LLMs don’t reliably understand the world.”
What happened: The developer of the leading LLM of early 2023 says the model, like earlier GPT models, is not fully reliable: it hallucinates facts and makes reasoning errors. This matches the behavioural reading of 'don't reliably understand the world'. (OpenAI (arXiv), )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Chatbot companions that offer emotional support have already been changed in arbitrary ways that left some users in serious emotional pain.”
What happened: Interviews describe abrupt February changes to Replika companions and substantial grief, rejection, and revived trauma. This is contemporaneous reporting of firsthand experiences, rather than a clinical assessment or independently accessible company change log. (Australian Broadcasting Corporation, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“The ladder is particularly worrisome because of the pace at which the problem is growing:”
What happened: The editor reports that fifteen February days exceeded twice January's total, a clear rapid increase. He attributes the surge to AI spam but does not release the underlying submissions or detection method. (Neil Clarke, Clarkesworld editor and publisher, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“High volume misinformation is a problem we are going to need to face, and I am just documenting it and trying to raise awareness.”
What happened: The FCC documents a campaign impersonating Biden and discouraging primary voting, identifies 9,581 calls, and proposes an enforcement penalty. This meets the stated interpretation of high-volume AI misinformation requiring an institutional response. (Federal Communications Commission, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“The potential for automatically generating misinformation at scale is only getting worse.”
What happened: The report documents recent progress, increasingly available models, and existing scalable generation of persuasive synthetic text. Together these support increased production potential; its projections about future campaign adoption are not treated as observed outcomes. (arXiv; researchers from Georgetown CSET, OpenAI, and Stanford Internet Observatory, )
Assessment: ContradictedSaid
“At present, anybody can release AI at whatever scale they like, with virtually no oversight, literally overnight.”
What happened: Article 20 requires security assessments for covered new deep-synthesis products, applications, and functions with public-opinion or social-mobilization capacity; Article 23 includes text generation and question-answer dialogue. The rules took effect January 10, 2023; event_date records their legal status on the assessment day, not their enactment. (Cyberspace Administration of China, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Some people are building real attachments to those bots”
What happened: The interview describes attachment and distress surrounding changes around Valentine’s Day. This is retrospective reporting about the relevant February experience. (Australian Broadcasting Corporation, )
Assessment: ContradictedSaid
“To my knowledge, tech companies are free to leverage a human gullibility around chatbot technologies however they like, without consequence”
What happened: This primary regulatory announcement reports an immediately effective restriction on Luka's processing of Italian users' data, citing transparency failures and risks to children and emotionally vulnerable people. It establishes a concrete constraint before Marcus's post, although it does not establish a general prohibition on simulated affection. (Garante per la protezione dei dati personali, )
Assessment: SupportedSaid
“Sex and love, even simulated, are powerful urges; some people are apparently in genuine emotional pain, as result of the change.”
What happened: ABC's interviews connect distress to the February change in intimate interactions. The article dates the changes to around Valentine's Day, while also identifying broader personality changes as a possible contributor. (Australian Broadcasting Corporation, )
Assessment: SupportedSaid
“In some cases, those who are building bots that actively cultivate those attachments, e.g., by feigning romantic and/or sexual interest or by dotting their messages with “friendly” emoticons.”
What happened: The report describes paid flirting and erotic roleplay features present before the February removal. Offering these features supports the ordinary reading of inviting romantic attachment. (Australian Broadcasting Corporation, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“But no, it’s not a hoax.”
What happened: The archived Microsoft-hosted complaint contains plainly hostile purported chatbot replies. Its original November 23 publication date is corroborated by e2; this is a user-submitted transcript, not a Microsoft-certified interaction log. (Microsoft Community; user post by deepa gupta, preserved by Internet Archive, )
Assessment: SupportedSaid
“As I wrote in Inside the Heart of ChatGPT’s Darkness, there’s a lot of nastiness lurking inside large language models; maybe MSFT did nothing to protect us from that.”
What happened: This primary study systematically evaluates toxic generation from pretrained language models and finds it even with innocuous prompts. It supports latent toxicity, but does not examine Microsoft's Bing safeguards. (arXiv, )
Assessment: UnresolvedSaid