Bubble Trouble
We recorded 23 claims from this post. 12 were checked against evidence. 11 can’t be checked. 11 more are held. Read the post at its source.
Claims from this post (23)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: ContradictedSaid
“In essence, the AI boom requires more high-quality data than currently exists to progress past the point we're currently at, which is one where the outputs of generative AI are deeply unreliable.”
What we tested: Generative AI cannot progress beyond its current deeply unreliable state without more high-quality data than currently exists.
What happened: SCoRe uses existing MATH and MBPP problems, generated correction traces, and correctness rewards. Its Gemini 1.0 Pro coding experiments improve self-correction on held-out HumanEval, demonstrating further error-correction progress without requiring an expanded stock of human-written material. (Google DeepMind researchers via arXiv, )
Assessment: Can’t be checkedSaid
“This is, of course, a terrible idea.”
What we tested: Training AI models on synthetic, AI-generated data is a terrible idea.
Assessment: SupportedSaid
“Models like ChatGPT and Claude are deeply dependent on training data to improve their outputs, and their very existence is actively impeding the creation of the very thing they need to survive.”
What we tested: Models like ChatGPT and Claude depend on training data and their existence is impeding the creation of new human data they need.
What happened: The study estimates a 25% relative reduction in Stack Overflow activity within six months of ChatGPT's release, using less-exposed comparison platforms. It examines content creation and reports no significant deterioration in post quality that would explain the loss as merely low-quality material. (PNAS Nexus; institutional record at UCL Discovery, )
Assessment: Can’t be checkedSaid
“Ironically, OpenAI's best hope for survival would be to fund as many news outlets as possible and directly incentivize them to do in-depth reporting, rather than proliferating a tech that unquestionably harms the media industry.”
What we tested: OpenAI's best hope for survival would be to fund as many news outlets as possible and directly incentivize them to do in-depth reporting, rather than proliferating a technology that harms the media industry.
Assessment: UnresolvedSaid
“Paying publishers to license their content doesn't actually fix the problem, because it doesn't increase the amount of content that they create, but rather helps line the pockets of executives and shareholders.”
What we tested: Licensing publisher content does not solve AI's data shortage because it does not increase the amount of content created.
Assessment: UnresolvedSaid
“That doesn't fix the overall problem that we are running out of data to feed the models, and that the solution in many cases is to use the broken tool to make more data.”
What we tested: Smaller models trained on limited synthetic datasets do not fix the overall problem that AI is running out of data to feed models, and that in many cases the solution is to use the broken tool to make more data.
Assessment: Can’t be checkedSaid
“The article is vague, likely because building Stargate is at this point somewhat theoretical.”
What we tested: The Information's article on Stargate is vague, likely because building Stargate is at this point somewhat theoretical.
Assessment: Can’t be checkedSaid
“Yet under the hood, outside of naked self-dealing, these firms may not be making much money from AI.”
What we tested: Outside self-dealing, Amazon, Microsoft and Google may not be making much money from AI.
Assessment: UnresolvedSaid
“Google and Amazon have invested billions in ChatGPT competitor Anthropic (and both claim to be Anthropic's primary cloud provider), and in doing so have guaranteed — according to a source of mine — $750 million a year (Google Cloud) and $800 million a year (AWS) in revenue by mandating that Anthropic uses their services to power its "Claude" model.”
What we tested: According to Zitron's source, Google and Amazon, through their investments in Anthropic, have guaranteed $750 million a year (Google Cloud) and $800 million a year (AWS) in revenue by mandating that Anthropic use their services to power Claude.
Assessment: Can’t be checkedSaid
“As Microsoft doesn't disclose Azure's actual revenue, it's unclear how much money this really is — but 6% of the year-over-year growth isn't exactly exciting.”
What we tested: Because Microsoft doesn't disclose Azure's revenue it is unclear how much money AI contributes, but AI accounting for 6% of Azure's year-over-year growth isn't exactly exciting.
Assessment: MixedSaid
“As I've discussed before, today's models are rife with hallucinations that make integrating them actively dangerous for many businesses, and the only way to have fewer hallucinations is to give the models more compute power to process and analyze data, a dwindling resource that AI is actively helping to destroy.”
What we tested: Today's models are rife with hallucinations that make integrating them actively dangerous for many businesses, and the only way to reduce hallucinations is to give models more compute power to process and analyze data, which is a dwindling resource that AI is actively helping to destroy.
What held: The January version reports frequent hallucinations on verifiable legal questions, supporting a material risk in consequential business workflows. It supports the risk clause within legal tasks, not a measured count of affected businesses or the compute-depletion claim. (arXiv; Matthew Dahl and colleagues, )
What did not: The paper reports improved Alpaca truthfulness on TruthfulQA through activation intervention and explains how to incorporate it into existing parameters. This counters the necessity of greater deployed processing capacity, although learning the intervention requires offline computation and the result is benchmark-specific. (arXiv; Kenneth Li and colleagues, )
Assessment: Can’t be checkedSaid
“Another hallmark of a bubble is a high-revenue, negative-profit industry. Glossy stories about OpenAI and Anthropic's multi-billion dollar revenues never seem to discuss profits, only endless growth.”
What we tested: A high-revenue, negative-profit industry is a hallmark of a bubble, and glossy stories about OpenAI's and Anthropic's multi-billion-dollar revenues never seem to discuss profits, only endless growth.
Assessment: Can’t be checkedSaid
“And when you cut through the hype, the AI industry is investing hundreds of billions of dollars to build infrastructure for a future that may never arrive based on the vague promise of hallucination-prone models.”
What we tested: The future that justifies the AI industry's hundreds of billions in infrastructure investment may never arrive.
Assessment: Can’t be checkedSaid
“Running Sora is likely deeply unprofitable, and reducing its generative hallucinations — monkeys with multiple left arms, for example — will require such a remarkable amount of resources that I'm not sure if it'll measurably improve in the next year.”
What we tested: Sora is likely deeply unprofitable and may not measurably improve its hallucinations within the next year.
Assessment: UnresolvedSaid
“Generative video models like OpenAI's Sora are dead on arrival, requiring massive amounts of compute and significantly more training data — which is itself both complex and scarce, and carries with it massive legal implications.”
What we tested: Generative video models like OpenAI's Sora are dead on arrival, because they require massive amounts of compute and significantly more training data, which is complex, scarce, and carries massive legal implications.
Assessment: UnresolvedSaid
“In the event that adoption slows — or, more accurately, demand never arrives — Microsoft, Amazon and Google only have so long before the markets sour on their "it's the future, we swear!" messaging, demanding real revenue to match their multi-billion dollar infrastructural bets.”
What we tested: If AI demand doesn't arrive, markets will eventually sour on Microsoft, Amazon and Google and demand real revenue for their AI infrastructure bets.
Assessment: ContradictedSaid
“Artificial intelligence isn't new, these models aren't new, and after just over a year of hype, generative AI is yet to prove it can do anything substantially different other than helping tech company CEOs raise money and create headlines.”
What we tested: Generative AI has not proven it can do anything substantially different besides helping CEOs raise money and make headlines.
What happened: The public research demonstration reported practical customer-support benefits beyond fundraising or publicity. The event date records when that demonstration became public, not when the underlying workplace observations occurred. (arXiv; Erik Brynjolfsson, Danielle Li, and Lindsey Raymond, )
Assessment: Can’t be checkedSaid
“What I fear is a cascade effect where artificial intelligence fails to provide businesses with anywhere near enough value to necessitate its costs, which will vastly reduce the amount of consumers and enterprises using or integrating models like ChatGPT, or using their own models on cloud providers like AWS or Azure.”
What we tested: AI may fail to provide enough business value to justify its costs, vastly reducing consumer and enterprise usage of models like ChatGPT.
Assessment: Can’t be checkedSaid
“This scenario is likely if the next generations of ChatGPT or Claude fail to make significant leaps in their capabilities, and as I've said above, their ability to do so is predicated on more training data and compute power than currently exists.”
What we tested: If next-generation ChatGPT or Claude fail to leap in capability, a cascade of reduced AI adoption is likely; such leaps require more data and compute than exists.
Assessment: UnresolvedSaid
“If businesses don't adopt AI at scale — not experimentally, but at the core of their operations — the revenue is simply not there to sustain the hype, and once the market turns, it will turn hard, demanding efficiency and cutbacks that will lead to tens of thousands of job cuts.”
What we tested: If businesses don't adopt AI at scale at the core of their operations, the revenue is not there to sustain the hype, and once the market turns it will turn hard, demanding efficiency and cutbacks that will lead to tens of thousands of job cuts.
Assessment: UnresolvedSaid
“Nvidia is likely to be the canary in the coal mine for the AI industry, and if AI fails to drive meaningful profits for both the customer and the provider, orders for Nvidia chips will dramatically slow.”
What we tested: Nvidia is likely to be the canary in the coal mine for the AI industry, and if AI fails to drive meaningful profits for both customers and providers, orders for Nvidia chips will dramatically slow.
Assessment: Can’t be checkedSaid
“This is why Sam Altman is so desperate to raise trillions to make a new chip company, or get Microsoft to build him a $100 billion supercomputer — because what he's promised isn't possible with today's technology, and may not be possible at all.”
What we tested: What Sam Altman has promised isn't possible with today's technology and may not be possible at all.
Assessment: UnresolvedSaid
“There is no iPhone moment coming, I'm afraid.”
What we tested: Generative AI will not have an iPhone-style breakthrough moment.
Held from this post (11)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews reached different assessments. Our reading, not assessed: When generative AI models are fed reams of content generated by other models, they learn rules from machine guesswork rather than communicated meaning, which makes that content somewhere between useless and actively harmful as training data.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: If Nvidia is the first to tumble, Zitron expects far worse things to happen to the cloud giants that bet tens or hundreds of billions of dollars on generative AI.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: Despite the hype, Zitron is seeing little evidence of actual AI adoption.
- The record does not meet the current rules. Our reading, not assessed: Zitron would bet money that ChatGPT was already trained on multiple Axel Springer publications, along with anything else publicly available on the web it could find.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: AI is reaching the point where physics, such as heat and electricity, is getting in the way of progressing much further, and where AI stands now is, in Zitron's judgment, fairly mediocre.
- The record does not meet the current rules. Our reading, not assessed: The AI boom drove global stock markets to their best first quarter in five years, and Zitron fears the boom is driven by a terrifyingly specious and unstable hype cycle.
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: There is no way to create enough data to train these models, and little suggests generative AI will make money for anyone but Nvidia.