Put Up Or Shut Up
We recorded 19 claims from this post. 7 were checked against evidence. 12 can’t be checked. 9 more are held. Read the post at its source.
Claims from this post (19)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
“I feel like the tech industry is currently in the midst of the most bizarre cognitive dissonance I've ever seen — more so than the metaverse, even — as company after company simply lies about their intentions and the power of AI.”
What we tested: The author feels that the tech industry is in the most bizarre cognitive dissonance he has ever seen, as company after company simply lies about its intentions and the power of AI.
Assessment: Can’t be checkedSaid
“There was nothing behind this, just like so much of the AI hype — a bunch of specious statements and empty promises in a wrapper of innovation despite creating nothing in the process.”
What we tested: Much of the AI hype is specious statements and empty promises that create nothing.
Assessment: ContradictedSaid
“To reach level 3 (whatever “Agents, systems that can take actions” means) you’d need a level of sentience dependent on entirely new forms of mathematics.”
What we tested: Reaching OpenAI's level 3 (agents) would require sentience dependent on entirely new forms of mathematics.
What happened: The account describes Claude Sonnet 3.7 using search, notes, Slack, and price-setting tools over a month, with humans providing physical services. I infer from this documented implementation that sustained action-taking did not require the asserted new mathematical foundations. (Anthropic, )
Assessment: Can’t be checkedSaid
“OpenAI is likely well aware that Large Language Models are incapable of reasoning, because Reuters reports that it’s got some sort of in-development model called Strawberry, which is designed to have human-like reasoning capabilities.”
What we tested: LLMs are incapable of reasoning, and OpenAI is likely aware of this.
Assessment: UnresolvedSaid
“I guarantee you Anthropic, Google and Meta are all working on something doing exactly the same thing, because what they are describing is something that limits every single Large Language Model due to the very nature of how they’re built, something Reuters even acknowledges by adding that “this is something that has eluded AI models to date.””
What we tested: Every LLM is limited by its nature in planning ahead and autonomously doing research, and Anthropic, Google and Meta are all working on the same problem.
Assessment: UnresolvedSaid
“The problem is that these pieces aren’t about breakthroughs — they’re marketing collateral dressed up as technological progress, superficially-exciting yet hollow on the inside, which almost feels a little too on the nose.”
What we tested: The stories about OpenAI's Strawberry and AGI levels are marketing collateral, not reports of breakthroughs.
Assessment: Can’t be checkedSaid
“I’d argue that the Bored Ape Yacht Club has a better chance of delivering, if only because “a blockchain game” would ostensibly use technology that exists.”
What we tested: The author would argue that the Bored Ape Yacht Club has a better chance of delivering its roadmap than OpenAI's Strawberry has of giving LLMs reasoning, if only because a blockchain game would ostensibly use technology that exists.
Assessment: Can’t be checkedSaid
“other than the fact that OpenAI said it cared a lot about safety and only sort-of did, if it really cared at all, which I don't believe it does.”
What we tested: OpenAI does not really care about safety.
Assessment: UnresolvedSaid
“The problem with stories like this is that they suggest that OpenAI is working on Artificial General Intelligence, or something that left unchecked could somehow destroy society, as opposed to what it’s actually working on — increasingly faster iterations of a Large Language Model that's absolutely not going to do that.”
What we tested: OpenAI is actually working on increasingly faster iterations of a Large Language Model, not on AGI or something that, left unchecked, could destroy society, and that LLM is absolutely not going to destroy society.
Assessment: Can’t be checkedSaid
“We don't need a story telling us it kind-of-sort-of-rushed a product to market, even though it didn't actually do that, but that this company, on a cultural level, operates without regard for the safety of its users, and is deliberately misrepresenting to outlets like Bloomberg that it is somehow on the path to creating AGI.”
What we tested: OpenAI operates without regard for user safety and deliberately misrepresents that it is on the path to AGI.
Assessment: UnresolvedSaid
“While there's tons of evidence that they're impossible to fix, surely a safety-minded culture would be one that sought to fix or mitigate the most obvious and dangerous problem of Large Language Models.”
What we tested: There is extensive evidence that LLM hallucinations are impossible to fix, and they are LLMs' most obvious and dangerous problem.
Assessment: Can’t be checkedSaid
“While there's tons of evidence that they're impossible to fix, surely a safety-minded culture would be one that sought to fix or mitigate the most obvious and dangerous problem of Large Language Models.”
What we tested: A safety-minded OpenAI should be investing in mitigating hallucinations.
Assessment: UnresolvedSaid
“they're allowed to get away from the very obvious problem: that Large Language Models are peaking, will not solve the kind of complex problems that actually matter, and OpenAI (and other LLM companies) are being allowed to accumulate money and power in a way that's allowed them to do actual damage in broad daylight.”
What we tested: LLMs are peaking in capability.
Assessment: Can’t be checkedSaid
“It claims it will "be trained on the best peer-reviewed science as well as Thrive's behavior change methodology," a mishmash of buzzwords and pseudoscientific pablum that means nothing because the company has produced no product and may likely never do so.”
What we tested: Thrive AI Health, the venture from the Thrive Global and OpenAI partnership, has produced no product and may likely never do so.
Assessment: Can’t be checkedSaid
“Generative AI's one real innovation is that it's allowed a certain class of scam artist to use the vague idea of "powerful automation" to hype companies to people that don't really know anything.”
What we tested: Generative AI's only real innovation is letting scam artists hype companies with vague promises of powerful automation.
Assessment: UnresolvedSaid
“The media seems nigh-on incapable of accepting that generative AI is a big, stupid, costly and environmentally-destructive bubble, to the point that they'll happily accept marketing slop and vague platitudes about how big something that's already here will be in the future based on a remarkable lack of proof.”
What we tested: Generative AI is a big, costly, environmentally destructive bubble.
Assessment: Can’t be checkedSaid
“And it’s time for the media to start pushing back and asking for real, tangible evidence that anything is actually happening.”
What we tested: The media should push back on AI claims and demand real, tangible evidence.
Assessment: Can’t be checkedSaid
“It’s time to treat OpenAI, Anthropic, Google, Meta and any other company pushing generative AI with suspicion, to see their acts as an investor-approved act of deception, theft and destruction.”
What we tested: OpenAI, Anthropic, Google, Meta and other generative AI companies should be treated with suspicion as engaged in deception, theft and destruction.
Assessment: Can’t be checkedSaid
“They’re not getting “reasoning,” nor are they getting “sentience,” nor are they “part of the path to superintelligence.””
What we tested: Generative AI models are not gaining reasoning or sentience and are not on the path to superintelligence.
Held from this post (9)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The record does not meet the current rules. Our reading, not assessed: ChatGPT spreads election misinformation, gives bomb-making and dangerous medical information, and generates buggy, vulnerable code.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: LLMs will not solve the kind of complex problems that actually matter.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: LLMs are currently incapable of the things needed to reach OpenAI's 'reasoner' level.
- The two reviews read the claim differently
- Attribution is uncertain
- The record does not meet the current rules. Our reading, not assessed: OpenAI's models damage the environment, provide false information, and do not make businesses any money.