They're Looting The Internet
We recorded 21 claims from this post. 9 were checked against evidence. 12 can’t be checked. 10 more are held. Read the post at its source.
Claims from this post (21)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
“As you'd expect, these lax standards have led to Facebook being flooded with content created with generative AI”
What we tested: Meta's lax moderation standards have led to Facebook being flooded with generative AI content.
Assessment: ContradictedSaid
“As a result, every single LLM-based application — ChatGPT, Google Gemini, Anthropic's Claude, Meta's LLaMA, to name but a few — is trained on an indeterminately-large portion of the internet made up of both Common Crawl (a 250-billion page repository of web content) and what appears to be as much of the rest of the internet as they're able to download, legally or otherwise.”
What we tested: Every single LLM-based application, including ChatGPT, Google Gemini, Anthropic's Claude and Meta's LLaMA, is trained on an indeterminately large portion of the internet made up of Common Crawl and what appears to be as much of the rest of the internet as they are able to download, legally or otherwise.
What happened: The methods restrict StarCoderBase's training set to The Stack's GitHub repositories; StarCoder adds Python fine-tuning from that corpus. This documented training recipe does not include Common Crawl. (BigCode research collaboration, arXiv, )
Assessment: ContradictedSaid
“These models are not "learning" or, let's be honest, "training" on this data, because that's not how they work — they're using mathematics to plagiarize it based on the likelihood that somebody else's answer is the correct one.”
What we tested: LLMs do not learn from data; they plagiarize it mathematically, going by the likelihood that someone else's answer is correct.
What happened: The documented objective is predictive model training. A probability over next tokens is not a probability that somebody else's complete answer is correct. (OpenAI, arXiv, )
Assessment: Can’t be checkedSaid
“While OpenAI, Google and Meta would like to claim that these are "publicly-available" works that they are "training on," the actual word for what they're doing is "stealing."”
What we tested: Although OpenAI, Google and Meta call it training on 'publicly-available' works, the right word for what they are doing is 'stealing'.
Assessment: Can’t be checkedSaid
“Every single tech company making a LLM is stealing, justifying it by using the previous model of the internet where everything published online was there for the taking and conflating access to content with ownership in the process.”
What we tested: Every single tech company making an LLM is stealing, justifying it with the earlier model of the internet in which everything published online was there for the taking, and conflating access to content with ownership.
Assessment: Can’t be checkedSaid
“LLMs are a globally-perpetuated act of theft taking place in broad daylight, to the point that OpenAI has told England's House of Lords that "it would be impossible to train today's leading AI models without using copyrighted materials."”
What we tested: LLMs are a globally perpetuated act of theft taking place in broad daylight.
Assessment: Can’t be checkedSaid
“The generative AI boom has exactly the same stench as the metaverse, and it's happening for exactly the same reason — the people making the products are not building things for human beings, but to show the markets that they'd continue to grow.”
What we tested: Like the metaverse, the generative AI boom is driven by the need to show markets continued growth rather than to build things for people.
Assessment: Can’t be checkedSaid
“OpenAI is no different. After all, it’s effectively a subsidiary of Microsoft.”
What we tested: OpenAI is effectively a subsidiary of Microsoft and, like the others, is a thief rather than an innovator.
Assessment: Can’t be checkedSaid
“And when you have an industry piloted by people who don't make products for people, you don't create anything useful, and you're doomed to make the same stupid, obvious mistakes.”
What we tested: When an industry is piloted by people who don't make products for people, it doesn't create anything useful and is doomed to make the same stupid, obvious mistakes.
Assessment: Can’t be checkedSaid
“Even the claims of what generative AI has done need to be taken with a grain — or a spoonful — of salt.”
What we tested: Claims about what generative AI has achieved should be treated with heavy skepticism.
Assessment: UnresolvedSaid
“First, AI can’t handle one of the most essential tasks of software engineering, which is collaboratively discussing the requirements of a project and how to implement the technology.”
What we tested: AI cannot collaboratively discuss project requirements and implementation, an essential software engineering task.
Assessment: Can’t be checkedSaid
“I’d say it has the engineering ability of a first-year CompSci undergraduate, but that would be unfair to first-year CompSci undergraduates.”
What we tested: Devin's engineering ability is below that of a first-year computer science undergraduate.
Assessment: UnresolvedSaid
Assessment: SupportedSaid
“As I wrote (and broadcast) recently, LLMs are running out of training data, and the desperation to find more will lead to significant legal consequences when it's revealed exactly how much they stole and from whom they stole it from.”
What we tested: The hunt for more training data will bring AI companies significant legal consequences once it is revealed how much they stole and from whom.
What happened: The order publicly disclosed the scale of acquisition and identified affected authors and works. The event date is the judicial disclosure, rather than the earlier downloads; the order also describes using subsets of the acquired collections to train LLMs. (U.S. District Court, Northern District of California, )
Assessment: UnresolvedSaid
“Yet I believe generative AI is going to lead several of these companies to ruin.”
What we tested: Generative AI will lead several big tech companies to ruin.
Assessment: UnresolvedSaid
“As these generative AI companies become more desperate, they'll (intentionally or otherwise) ingest AI-generated content, which causes a degenerative training effect called "model collapse" (also known as Habsburg AI) that destroys the model's ability to deliver useful answers.”
What we tested: As generative AI companies grow desperate for data, they will ingest AI-generated content, causing model collapse that destroys their models' usefulness.
Assessment: UnresolvedSaid
“LLMs also tend to hallucinate, a virtually-unsolvable problem where they authoritatively make incorrect statements that creates horrifying results in generative art and renders them too unreliable for any kind of mission critical work.”
What we tested: LLM hallucination is a virtually unsolvable problem that makes LLMs too unreliable for any mission-critical work.
Assessment: Can’t be checkedSaid
“Generative AI's energy demands are unsustainable, as are its compute demands”
What we tested: Generative AI's energy and compute demands are unsustainable.
Assessment: Can’t be checkedSaid
“The answer is obvious: because they believe it'll make them money.”
What we tested: The tech industry is pushing generative AI on people because it believes generative AI will make it money.
Assessment: Can’t be checkedSaid
“Yet I believe that they're all disconnected from any actual value creation.”
What we tested: He believes Google, Meta, Microsoft and OpenAI are all disconnected from any actual value creation.
Assessment: Can’t be checkedSaid
“Nothing — and I repeat, nothing — good will come from this.”
What we tested: Nothing good will come from Google's leadership shuffle that moved senior AI engineer Cheenu Venkatachary to lead the ranking and search quality teams.
Held from this post (10)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: Google has not found a way to make money from its generative search product, which still hallucinates, and is thinking about charging for it.
- The two reviews reached different assessments. Our reading, not assessed: In its demonstrated Upwork task, Devin produced code that missed requirements, used outdated approaches, and took longer than a human would.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: By all accounts, generative AI companies are already ingesting AI-generated content, as when Adobe's Firefly AI accidentally ingested generative art that had found its way into the Adobe Stock image pool.
- The record does not meet the current rules. Our reading, not assessed: Despite all the hype, AI neither solves any real problems nor actually makes these companies (Google, Meta, Microsoft and OpenAI) any money.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The two reviews read the claim differently