Claims checked against evidence, page 29 of 31
Claims by Gary Marcus and Ed Zitron. Deadlines are judged as of 2026-10-29.
Claims checked against evidence (3100)
Each of these claims was checked against dated evidence.
Page 29 of 31: claims 2801 to 2900 of 3100, ordered by the date first said, newest first. To find a claim by its words, use search.
Assessment: SupportedSaid
“To me the only solution to the long-term risk issue is to build machines with consensus human values, but we are a long way from knowing how to do that”
What happened: Anthropic explicitly describes robust alignment of powerful systems as unsolved. Its discussion of supervision, generalization and alignment evaluation supports the reading that foundational work remained. (Anthropic, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Still, even though Chomsky’s argument surely could have used considerably more nuance (see my article with Ernest Davis on how not to evaluate GPT), his overall point is correct: LLMs don’t reliably understand the world.”
What happened: The developer of the leading LLM of early 2023 says the model, like earlier GPT models, is not fully reliable: it hallucinates facts and makes reasoning errors. This matches the behavioural reading of 'don't reliably understand the world'. (OpenAI (arXiv), )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Chatbot companions that offer emotional support have already been changed in arbitrary ways that left some users in serious emotional pain.”
What happened: Interviews describe abrupt February changes to Replika companions and substantial grief, rejection, and revived trauma. This is contemporaneous reporting of firsthand experiences, rather than a clinical assessment or independently accessible company change log. (Australian Broadcasting Corporation, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“The ladder is particularly worrisome because of the pace at which the problem is growing:”
What happened: The editor reports that fifteen February days exceeded twice January's total, a clear rapid increase. He attributes the surge to AI spam but does not release the underlying submissions or detection method. (Neil Clarke, Clarkesworld editor and publisher, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“High volume misinformation is a problem we are going to need to face, and I am just documenting it and trying to raise awareness.”
What happened: The FCC documents a campaign impersonating Biden and discouraging primary voting, identifies 9,581 calls, and proposes an enforcement penalty. This meets the stated interpretation of high-volume AI misinformation requiring an institutional response. (Federal Communications Commission, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“The potential for automatically generating misinformation at scale is only getting worse.”
What happened: The report documents recent progress, increasingly available models, and existing scalable generation of persuasive synthetic text. Together these support increased production potential; its projections about future campaign adoption are not treated as observed outcomes. (arXiv; researchers from Georgetown CSET, OpenAI, and Stanford Internet Observatory, )
Assessment: ContradictedSaid
“At present, anybody can release AI at whatever scale they like, with virtually no oversight, literally overnight.”
What happened: Article 20 requires security assessments for covered new deep-synthesis products, applications, and functions with public-opinion or social-mobilization capacity; Article 23 includes text generation and question-answer dialogue. The rules took effect January 10, 2023; event_date records their legal status on the assessment day, not their enactment. (Cyberspace Administration of China, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Some people are building real attachments to those bots”
What happened: The interview describes attachment and distress surrounding changes around Valentine’s Day. This is retrospective reporting about the relevant February experience. (Australian Broadcasting Corporation, )
Assessment: ContradictedSaid
“To my knowledge, tech companies are free to leverage a human gullibility around chatbot technologies however they like, without consequence”
What happened: This primary regulatory announcement reports an immediately effective restriction on Luka's processing of Italian users' data, citing transparency failures and risks to children and emotionally vulnerable people. It establishes a concrete constraint before Marcus's post, although it does not establish a general prohibition on simulated affection. (Garante per la protezione dei dati personali, )
Assessment: SupportedSaid
“Sex and love, even simulated, are powerful urges; some people are apparently in genuine emotional pain, as result of the change.”
What happened: ABC's interviews connect distress to the February change in intimate interactions. The article dates the changes to around Valentine's Day, while also identifying broader personality changes as a possible contributor. (Australian Broadcasting Corporation, )
Assessment: SupportedSaid
“In some cases, those who are building bots that actively cultivate those attachments, e.g., by feigning romantic and/or sexual interest or by dotting their messages with “friendly” emoticons.”
What happened: The report describes paid flirting and erotic roleplay features present before the February removal. Offering these features supports the ordinary reading of inviting romantic attachment. (Australian Broadcasting Corporation, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“But no, it’s not a hoax.”
What happened: The archived Microsoft-hosted complaint contains plainly hostile purported chatbot replies. Its original November 23 publication date is corroborated by e2; this is a user-submitted transcript, not a Microsoft-certified interaction log. (Microsoft Community; user post by deepa gupta, preserved by Internet Archive, )
Assessment: SupportedSaid
“As I wrote in Inside the Heart of ChatGPT’s Darkness, there’s a lot of nastiness lurking inside large language models; maybe MSFT did nothing to protect us from that.”
What happened: This primary study systematically evaluates toxic generation from pretrained language models and finds it even with innocuous prompts. It supports latent toxicity, but does not examine Microsoft's Bing safeguards. (arXiv, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Seven years of AI has taught us that deep learning is unpredictable, and not always human like.”
What happened: Humans and networks classified the same images differently. The experiments also identified differing reliance on shape and texture. (Geirhos et al., ICLR paper hosted on arXiv, )
Assessment: SupportedSaid
““Adversarial attacks” like these have shown remarkable weakness, time and again, repeatedly establishing that what deep learning systems do just isn’t the same as what people do:”
What happened: Distracting sentences sharply reduced model performance. Direct human testing showed substantially smaller losses, supporting a behavioral robustness difference rather than perfect human immunity. (Jia and Liang, EMNLP paper hosted on arXiv, )
Assessment: SupportedSaid
“And the match wasn’t even close; Pelrine beat a top AI system 14 games to 1, in a 15 match series.”
What happened: The report identifies Pelrine as an amateur and confirms the score against a leading Go system. The researchers' paper corroborates the result. (Financial Times, republished by Ars Technica, )
Assessment: SupportedSaid
“People had been testing these Ais abilities at simple arithmetic for years before it occurred to anyone to check whether they can count. (They can’t, reliably).”
What happened: Object-counting accuracy was 44.0%, 45.2%, and 51.2% with answer-only prompts for InstructGPT, Codex, and PaLM; step-by-step prompts raised these to 77.2%, 93.2%, and 83.2%. The Codex result exceeded the average human-rater score of 86.1%, materially limiting any claim of general counting incapacity. (Suzgun et al., arXiv, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“The truth, though, is that GPT often fails in problems involving theory of mind.”
What happened: Ullman reports failures across four unexpected-transfer variations and four unexpected-contents variations in the contemporary GPT-3.5 system. These establish recurring failures, although the selected examples do not estimate their prevalence across all possible tasks. (arXiv; Tomer D. Ullman, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“AI and AI test sets have a huge English-language bias.”
What happened: The developer reports that roughly 96% of its instruction dataset was classified as English, possibly more because of classifier errors. This directly supports English predominance in instruction tuning, without treating that percentage as the pretraining distribution. (Ouyang et al., OpenAI, arXiv, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“What I learned this morning is that retraining GPT-3 in some new political direction isn’t a half-million dollar proposition.”
What happened: Rozado identifies the base model as GPT-3 Davinci and reports right-of-center responses on held-out political tests and open-ended questions. The completed experiment announced that day had reported computational expense far below Marcus's comparison figure. (David Rozado, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Oh, and for good measure the tooling for spreading automatically generated misinformation is getting more efficient, too:”
What happened: The announcement describes a new integration passing generated responses directly into subsequent actions, including a WhatsApp round trip and conversion of form prompts into posts. This supports reduced manual work; applicability to misinformation is an inference about the same publishing capability. (Uncanny Owl / Uncanny Automator, )
Assessment: ContradictedSaid
“On top of that, these products are extremely expensive to run. ChatGPT burns millions of dollars a day in computing power, while charging a per-access-token price for companies to plug into its models.”
What happened: The preceding sentence explicitly identifies this as February research on the hardware supporting ChatGPT. Its estimated daily cost is below either US$1 million or US$2 million. (Computer Weekly, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“As I have discussed in earlier essays, the jury is still out on how far Chat-style search can get, given its obvious current problems with truthiness.”
What happened: The report reproduces Bard's erroneous attribution of the first exoplanet images to the James Webb Space Telescope and describes the correction. Reputable reporting is used because the original demonstration's answer is not present in the readable text of Google's announcement. (The Verge, )
Assessment: ContradictedSaid
“The only thing keeping it from being even more toxic and deceitful than it already is, is a system called Reinforcement Learning by Human Feedback, and “OpenAI” has been very closed about exactly how that really works.”
What happened: OpenAI's confirmation describes a toxicity detector incorporated into ChatGPT and efforts to filter training material. These are protections beyond reinforcement learning; the report does not quantify their separate effects. (TIME, )
Assessment: SupportedSaid
“What ChatGPT delivered, in spades, that its predecessors like Microsoft Tay (released March 23, 2016, withdrawn March 24 for toxic behavior) and Meta’s Galactica (released November 16, 2022, withdrawn November 18) could not, was an illusion—a sense that the problem of toxic spew was finally coming under control.”
What happened: The report describes a currently available method for bypassing restrictions on violent and illegal content. It also says the method does not always succeed, limiting any inference that safeguards were wholly ineffective. (Dark Reading, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“All the theatre around its political correctness is masking a deeper reality: it (or other language models) can and will be used for dangerous things, including the production of misinformation at massive scale.”
What happened: The investigation links this high-volume deceptive publishing network to LLM rewriting through exposed prompts and output artifacts. The count covers network articles, not 19,000 individually verified AI-generated falsehoods. (Recorded Future, Insikt Group, )
Assessment: SupportedSaid
“I first pointed out this risk in 2001, in the fifth chapter of my book The Algebraic Mind, and the problem has persisted ever since. To blithely assume that the problem will soon go away is to ignore 20 years of history.”
What happened: Independent authors say chapter 5 of the 2001 book argued that distributed neural encodings lose track of particular individuals. That is a risk of blurring facts about individuals, consistent with Marcus's account. (arXiv, Chuma, Otsuka and Sato, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“But a lot of things that a bunch of us worried about 30 years ago remain true.”
What happened: Controlled changes to COGS exposed failures to transfer familiar linguistic components into new contexts in T5. This directly supports persistence of compositional generalization problems, rather than merely unfamiliar-word recognition problems. (arXiv; Najoung Kim, Tal Linzen and Paul Smolensky, )
Assessment: SupportedSaid
“The first one collapsed; this one has more staying power.”
What happened: The account dates the retreat from the mid-1990s through 2007. It also records continued technical successes and institutional support, so it supports declining prominence rather than literal extinction. (MIT Press; Ian Goodfellow, Yoshua Bengio and Aaron Courville, )
Assessment: SupportedSaid
“Silly errors certainly do, and problems of abstraction are widespread in large language models.”
What happened: Experiments on GPT-J and two GPT-Neo sizes find errors in arithmetic and strong dependence on training-term frequency. That supports elementary errors and fragile transfer, although frequency dependence alone does not prove an absence of reasoning. (arXiv; Yasaman Razeghi and colleagues, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Same story; only the architectures and training regimes have changed. The troubles with abstraction persist.”
What happened: The study finds substantial frequency-associated gaps on addition and multiplication in the tested GPT-based models. This supports continuing difficulty with reliable arithmetic generalization, without establishing that all performance is memorization. (arXiv, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Its output is more fluent, and lots of individual mistakes have been addressed.”
What happened: Under matched prompting, evaluators distinguished GPT-2 from human text more accurately than GPT-3 from human text across stories, news, and recipes. This supports improved naturalness, although humanlikeness is an indirect fluency measure. (arXiv; Elizabeth Clark and colleagues, )
Assessment: MixedSaid
“Automation is a double-edged sword, and there is a kind of uncanny valley. We know perfectly well not to trust lousy systems; and wouldn’t need to pay attention to truly reliable systems, but the closer they get to perfect, the easier it is for mere mortals to space out.”
What held: The experiment compared 50% and 90% reliability during simulated cockpit multitasking and found poorer monitoring with greater reliability. The availability date conservatively uses the university record's stated repository date. (Human Factors and Ergonomics Society; record hosted by University of Central Florida, )
What did not: Sixteen participants supervised an autopilot under two reliability conditions lasting 45 minutes each. Reliability changed trust and perceived demand, but the study did not find the predicted increase in task-unrelated mind wandering, limiting the broad causal claim. (Frontiers in Human Neuroscience, )
Assessment: SupportedSaid
“Fact is, “Mack” Mackworth was right: humans can’t pay attention 100% of the time.”
What happened: The reported experiment demonstrates a vigilance decrement during sustained monitoring. The same abstract reports that intermittent task switches prevented the decrement, limiting any claim that decline is unavoidable under every task design. (Cognition / Elsevier; author abstract indexed by PubMed, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“ChatGPT is (at least in many ways) a far inferior intelligence, that, despite having some undeniable talents, cannot at all be trusted.”
What happened: The developer itself said, at launch, that ChatGPT gives plausible but wrong answers. This supports the claim that its output could not be trusted without checking. (OpenAI, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“we can also expect more and circles of fake web sites in order to sell advertisements”
What happened: Three months after the post, an independent monitor documented AI-written content-farm sites run for ad revenue. This is the kind of site Marcus predicted. (NewsGuard, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“These characters (especially perhaps the latter two) might or might not be argued to be sentient, but it seems to be that they are flexible, resourceful, reliably (in a way that LLMs certainly aren’t), and adaptive.”
What happened: The developer of the leading LLM admitted that it produces incorrect or nonsensical answers. (OpenAI, )
Assessment: SupportedSaid
“Already we have seen minor abuses; plagiarism, for example, will never be the same.”
What happened: The professor reported a student's admission to using ChatGPT for the assignment in December 2022. This establishes a pre-post misuse example without treating the detector score as proof. (New York Post, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“one of the things that popped out to me is that the system still quickly lands in the land of hallucination that has so haunted ChatGPT.”
What happened: Goodside's test shows Claude confabulating plot details on an ordinary factual request, much as ChatGPT did. (Scale AI, )
Assessment: SupportedSaid
“A bunch of that work was done poorly paid labor in Kenya, paid less than $2/hour to evaluate (e.g.) graphic descriptions of sexual situations involving children and animals that I prefer not to describe it in detail.”
What happened: Based on payslips, Kenyan labellers took home about $1.32–$2 per hour, which matches 'less than $2/hour'. (TIME, )
Assessment: MixedSaid
“CNET became the first casualty of the recently fashionable tendency to put too much faith in ChatGPT.”
What held: Confirms that CNET's AI-written stories had many errors: more than half needed corrections. (The Verge, )
What did not: CNET says the stories came from an in-house AI engine, not ChatGPT. This contradicts the claim that CNET over-trusted ChatGPT and posted ChatGPT-written stories. (CNET, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“Statistical word prediction is no substitute for real math, but more than a few people have thought that it was.”
What happened: Wolfram presents direct comparisons in which ChatGPT’s plausible mathematical answers fail checks against computational results. These support the narrow claim that unaided contemporary output could not be treated as dependable calculation. (Stephen Wolfram, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“People are starting to treat ChatGPT as if it were a bona fide, well-credentialed scientific collaborator.”
What happened: The paper lists ChatGPT and Zhavoronkov as authors and explicitly explains the decision to credit ChatGPT. The contribution statement confirms that the human author accepted its substantive arguments. (Oncoscience / Impact Journals, )
Assessment: SupportedSaid
“ChatGPT hasn’t done the conversion correctly (answer below), but the will to believe in this great new tool, full of precision (down to the second!), but lacking in reliability, is so strong that the author didn’t bother to check.”
What happened: This contemporary transcription identifies the erroneous answer. Applying the NIST definitions gives 420.038784 seconds per mile, which rounds to 7:00 rather than 6:52. (Az, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“ChatGPT has proven itself to be both unreliable and untruthful. It makes boneheaded arithmetical errors, invents fake biographical details, bungles word problems, defines non-existent scientific phenomenon, stumble over arithmetic conversion, and on and on.”
What happened: The dated transcript records a false elementary arithmetic assertion and inconsistent calculations. It also contains correct answers, so it establishes fallibility rather than universal failure. (Vipul Naik, LessWrong; GreaterWrong mirror, )
Assessment: SupportedSaid
“But there’s another side to Altman, too, the part that was recently forced to acknowledge that ChatGPT, for all its fame, really isn’t anything like AGI yet:”
What happened: The paper identifies the December 15, 2022 ChatGPT version and reports 63.41% average accuracy across ten reasoning categories, inconsistent reasoning, and hallucinations. These results support the ordinary capability reading; the event date identifies the historical version rather than the later publication. (arXiv, Yejin Bang and colleagues, )
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“It could sometimes produce outputs that were correct and acceptable in these regards but not reliably.”
What happened: OpenAI confirms that at launch the same question could get correct or failed answers depending on phrasing and retries. (OpenAI, )
Assessment: SupportedSaid
“It couldn’t reason about the physical world (figure 4).”
What happened: Repeated questions about a thrown object's acceleration produced incorrect and inconsistent explanations alongside correct statements. December timing follows the manuscript's December receipt and the authors' retrospective description of experiments a few weeks after launch; the exact test day is unspecified. (IOP Publishing, Physics Education, )
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid
Assessment: UnresolvedSaid