Deep Impact
We recorded 50 claims from this post. 27 were checked against evidence. 23 can’t be checked. 23 more are held. Read the post at its source.
Claims from this post (50)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
“In an AI context, reasoning works by breaking down a prompt into a series of different steps with "considerations" of different approaches — effectively a Large Language Model checking its work as it goes, with no thinking involved, because these models do not "think" or "know" stuff.”
What we tested: AI reasoning models do not think or know anything; reasoning is an LLM checking its work step by step.
Assessment: UnresolvedSaid
“And, as I noted at the time, it was not particularly reliable, failing to accurately count the number of times the letter ‘r’ appeared in the word “strawberry.””
What we tested: OpenAI's o1 was not particularly reliable at launch, failing to count the r's in 'strawberry'.
Assessment: Can’t be checkedSaid
“At this point, it's fairly obvious that OpenAI wasn’t anywhere near the “forefront of AI development,” and now that its competitive advantage is effectively gone, there are genuine doubts about what comes next for the company.”
What we tested: It is fairly obvious that OpenAI wasn't anywhere near the forefront of AI development, its competitive advantage is now effectively gone, and there are genuine doubts about what comes next for the company.
Assessment: SupportedSaid
“DeepSeek's models don't require massive new data centers (they run on the GPUs currently used to run services like ChatGPT, and can even work on more austere hardware), nor do they require an endless supply of bigger, faster NVIDIA GPUs every year to progress.”
What we tested: DeepSeek's models do not require massive new data centers or ever-bigger NVIDIA GPUs to progress.
What happened: The report identifies the hardware actually used and describes training-efficiency improvements and deployment on H800s. It supports the existing-hardware route, although it is a company report and does not document the construction history of the facility. (DeepSeek-AI, )
Assessment: UnresolvedSaid
“Once these models are hosted elsewhere, and once DeepSeek's methods (which I'll get to shortly) are recreated (which won't take long), I believe we'll see that these prices are indicative of how cheap these models are to run.”
What we tested: Once DeepSeek's models are hosted elsewhere and its methods recreated (soon), its low prices will prove to reflect genuinely low running costs.
Assessment: UnresolvedSaid
“I do not believe that the companies making foundation models (such as OpenAI and Anthropic) have been incentivized to do more with less, and because their chummy relationships with hyperscalers were focused almost entirely on "make the biggest, most hugest models possible, using the biggest, most hugest chips," and because the absence of profitability didn’t stop them from raising more money, efficiency was never a major problem for them.”
What we tested: He hypothesises that foundation model companies such as OpenAI and Anthropic were not incentivised to do more with less, and that because their hyperscaler relationships focused almost entirely on the biggest models and chips, and because being unprofitable didn't stop them raising money, efficiency was never a major problem for them.
Assessment: Can’t be checkedSaid
“As I've written before, OpenAI simply burns money, has been allowed to burn money, and up until recently likely would've been allowed to burn even more money, because everybody — all of the American model developers — appeared to agree that the only way to develop Large Language Models was to make the models as big as humanly possible, and work out troublesome stuff like "making them profitable" later”
What we tested: OpenAI burns money, has been allowed to, and until recently likely would have been allowed to burn even more, because all American model developers appeared to agree that the only way to develop LLMs was to make them as big as possible and work out profitability later.
Assessment: Can’t be checkedSaid
“Synthetic data might work where the output is something that you could figure out on a TI-83 calculator, but when you get into anything a bit more fuzzy (like written text, or anything with an element of analysis) you’ll likely start to encounter unhappy side effects..”
What we tested: Synthetic training data may work for unambiguous domains like math but will likely cause bad side effects for fuzzy domains like text or analysis.
Assessment: Can’t be checkedSaid
“This doesn’t mean that we know for a fact that DeepSeek distilled 4o or Claude, but frankly, it would be odd if they didn’t.”
What we tested: DeepSeek likely distilled GPT-4o or Claude, though this is not known for certain.
Assessment: UnresolvedSaid
“As I've written about many, many, many times, the Large Language Models run by companies like OpenAI, Anthropic, Google and Meta are unprofitable and unsustainable, and the transformer-based architecture they run on has peaked.”
What we tested: LLMs run by OpenAI, Anthropic, Google and Meta are unprofitable and unsustainable.
Assessment: UnresolvedSaid
“As I've written about many, many, many times, the Large Language Models run by companies like OpenAI, Anthropic, Google and Meta are unprofitable and unsustainable, and the transformer-based architecture they run on has peaked.”
What we tested: The transformer-based architecture underlying LLMs has peaked.
Assessment: MixedSaid
“They're running out of training data, and the actual capabilities of these models were peaking as far back as March 2024.”
What we tested: LLM developers are running out of training data, and model capabilities had been peaking since March 2024.
What held: The estimate supports an approaching constraint on usable human-generated public text under continuing scaling trends. It does not show that data was already exhausted, and the discussion identifies other data sources and ways progress could continue. (Epoch AI, )
What did not: Using its baseline testing framework, ARC Prize reports public-evaluation scores of 21.2% for o1-preview and 9% for GPT-4o. This is post-March capability progress, although the evaluator also reports substantially greater inference time. (ARC Prize, )
Assessment: UnresolvedSaid
“After all, OpenAI lost $5 billion (after $3.7 billion in revenue, too!), and Anthropic just under $3 billion in 2024.”
What we tested: In 2024 OpenAI lost $5 billion on $3.7 billion revenue and Anthropic lost just under $3 billion.
Assessment: Can’t be checkedSaid
“My mistake was — if you can believe this — being too generous to the AI companies, assuming that they didn’t pursue efficiency because they couldn’t, and not because they couldn’t be bothered.”
What we tested: He says he was too generous in assuming that AI companies didn't pursue efficiency because they couldn't, when potentially they didn't because they couldn't be bothered.
Assessment: ContradictedSaid
“The current models developed by both the hyperscalers (Gemini, Llama, et. al) and multi-billion-dollar "startups" like OpenAI and Anthropic are horribly inefficient, I had just made the mistake of assuming that they'd actually tried to make them more efficient.”
What we tested: Current models from Google, Meta, OpenAI and Anthropic are horribly inefficient, and their makers never really tried to make them efficient.
What happened: Meta describes concrete changes to parallelization, reliability, storage and training infrastructure, alongside inference optimizations. This is substantive efficiency engineering, although the improvement magnitude is company-reported. (Meta, )
Assessment: Can’t be checkedSaid
“Nevertheless, OpenAI and Anthropic both have enough money and hiring power to have tried — and succeeded — in creating a model this efficient and capable of running on older GPUs, except what they actually wanted was more rapacious growth and the chance to build even bigger data centers with even more compute.”
What we tested: OpenAI and Anthropic had the resources to build a model as efficient as DeepSeek's but preferred growth and bigger data centers.
Assessment: Can’t be checkedSaid
“OpenAI has pledged $19 billion to fund the "Stargate" data center — an amount it is somehow going to raise through further debt and equity raises, despite the fact that it’s likely already in the process of raising another round as we speak just to keep the company afloat.”
What we tested: OpenAI is likely already raising another funding round just to stay afloat, on top of its $19 billion Stargate pledge.
Assessment: MixedSaid
“The launch of its "Operator" "agent" was a joke — a barely-functional product that is allegedly meant to control your computer and take distinct actions, but doesn't seem to work.”
What we tested: OpenAI's Operator agent is barely functional and does not seem to work.
What held: The tester reports five failures involving web research and online tools. These show real launch limitations, but the small, relatively demanding task set is not a representative consumer success-rate estimate. (Zengyi Qin / Hugging Face Community, )
What did not: Newton describes successful browser research producing walking-tour links and a completed lesson-plan task. His same review documents substantial shopping friction, so these successes limit the broad dismissal without establishing reliable autonomy. (Platformer / Casey Newton, )
Assessment: Can’t be checkedSaid
“Let's be frank: these companies aren't building shit. OpenAI and Anthropic are both limply throwing around the idea that "agents are possible" in an attempt to raise more money to burn, and after the launch of DeepSeek, I have to wonder what any investor thinks they're investing in.”
What we tested: OpenAI and Anthropic 'aren't building shit' and are limply floating the idea that agents are possible in an attempt to raise more money to burn, leaving him to wonder after DeepSeek what investors think they are investing in.
Assessment: UnresolvedSaid
“And this, in turn, will make any investor think twice about writing the company a blank check — which, as I’ve said ad nauseum, is potentially fatal, as OpenAI needs to continually raise more money than any startup ever has in history, and it has no path to breaking even.”
What we tested: If OpenAI 'adds on' DeepSeek's approach, conceding that it slipped behind, any investor will think twice about writing it a blank check, which is potentially fatal because OpenAI must keep raising more money than any startup in history and has no path to breaking even.
Assessment: Can’t be checkedSaid
“If OpenAI wants to do its own cheaper, more-efficient model, it’ll likely have to create it from scratch, and while it could do distillation to make it "more OpenAI-like" using OpenAI's own models, that's effectively what DeepSeek already did.”
What we tested: If OpenAI wants a cheaper, efficient model, it will likely have to build it from scratch.
Assessment: Can’t be checkedSaid
“Even with OpenAI's much larger team and more powerful hardware, it's hard to see how creating a smaller, more-efficient, and almost-as-powerful version of o1 benefits the company, because said version has, well, already been beaten to market by DeepSeek, and thanks to DeepSeek will almost certainly have a great deal of competition for a product that, to this day, lacks any real killer apps.”
What we tested: If OpenAI builds a smaller, more efficient, almost-as-powerful o1, it is hard to see how that benefits the company, because DeepSeek has already beaten it to market and it will almost certainly face a great deal of competition for a product that still lacks killer apps.
Assessment: UnresolvedSaid
“He ended with “look forward to bringing you all AGI and beyond” — something which, I add, has always been close on the horizon in Altman’s world, although curiously has yet to materialize, or even come close to materializing.”
What we tested: AGI, which Altman has always portrayed as close on the horizon, has yet to materialize or even come close to materializing.
Assessment: SupportedSaid
“Whether or not someone chooses to pay DeepSeek is largely irrelevant — someone else will take what it’s created and build their own, or people will start running their own DeepSeek instances renting GPUs from one of the various cloud computing firms.”
What we tested: Others will build their own models from DeepSeek's work or run their own DeepSeek instances on rented cloud GPUs.
What happened: Skywork records the release of its final OR1 models on May 13 and explicitly identifies DeepSeek's distilled models as their starting points. This is an actual third-party derivative release after the forecast. (Skywork AI, )
Assessment: UnresolvedSaid
“While NVIDIA will find other ways to make money — Jensen Huang always does — it's going to be a hard sell for any hyperscaler to justify spending billions more on GPUs to markets that now know that near-identical models can be built for a fraction of the cost with older hardware.”
What we tested: Hyperscalers will struggle to justify spending billions more on GPUs to markets now aware near-identical models can be built cheaply on older hardware.
Assessment: UnresolvedSaid
“After all, OpenAI loses money on every transaction — even its $200-a-month "ChatGPT Pro" subscription.”
What we tested: OpenAI loses money on every transaction, including the $200/month ChatGPT Pro subscription.
Assessment: UnresolvedSaid
“And if OpenAI cuts its prices to compete with DeepSeek, its losses will only deepen.”
What we tested: If OpenAI cuts prices to compete with DeepSeek, its losses will deepen.
Assessment: Can’t be checkedSaid
“It still isn't clear if these models are going to be profitable — as discussed, it's unclear who funds DeepSeek and whether its current pricing is sustainable — but they are likely going to be a damn sight more profitable than anything OpenAI is flogging.”
What we tested: DeepSeek's models are likely to be much more profitable than OpenAI's products, though their profitability is uncertain.
Assessment: Can’t be checkedSaid
“Putting aside my personal beliefs for a second, it’s fairly obvious why these companies wouldn’t want to create something like DeepSeek — because creating an open source model that uses less resources means that OpenAI, Anthropic and their associated hyperscalers would lose their soft monopoly on Large Language Models.”
What we tested: It is fairly obvious that OpenAI, Anthropic and their hyperscalers wouldn't want to create something like DeepSeek, because an efficient open-source model would cost them their soft monopoly on LLMs.
Assessment: UnresolvedSaid
“We also know that OpenAI’s training and inference costs in 2024 were around $7 billion, meaning that either refining current models or building new ones is quite costly.”
What we tested: OpenAI's training and inference costs in 2024 were around $7 billion.
Assessment: UnresolvedSaid
“This isn’t a particularly exciting pitch to investors or the public markets, because there’s no product, innovation or business model to point to, and if they’d actually try and productize it and turn it into a business, it’s quite obvious at this point that there really isn’t a multi-trillion dollar industry for generative AI.”
What we tested: It is quite obvious at this point that if LLM companies actually tried to productize LLMs and turn them into a business, there really isn't a multi-trillion-dollar industry for generative AI.
Assessment: Can’t be checkedSaid
“Large Language Models are, in and of themselves, a fairly bland software product, which is why we’re yet to see any killer apps.”
What we tested: LLMs are a bland software product, which is why no killer apps have emerged.
Assessment: UnresolvedSaid
“OpenAI only makes 27% of its revenue from selling access to its models — around a billion dollars in annual recurring revenue — with the rest ($2.7 billion or so) coming from subscriptions to ChatGPT.”
What we tested: OpenAI makes only 27% of its revenue, around $1 billion in annual recurring revenue, from selling access to its models, with the rest (about $2.7 billion) coming from ChatGPT subscriptions.
Assessment: Can’t be checkedSaid
“Even OpenAI’s push into the federal government, with the release of ChatGPT Gov, is unlikely to reverse its dismal fortunes.”
What we tested: ChatGPT Gov is unlikely to reverse OpenAI's dismal financial fortunes.
Assessment: Can’t be checkedSaid
“Post-DeepSeek, there isn’t really a compelling argument for investing hundreds of billions of capital expenditures in data centers, buying new GPUs, or even pursuing Large Language Models as they currently stand.”
What we tested: After DeepSeek, there is no compelling argument for hundreds of billions in data center capex, new GPUs, or current-style LLMs.
Assessment: Can’t be checkedSaid
“I’d argue that despite the significant price difference between o1 and DeepSeek’s r1 reasoning model, the real danger to both OpenAI and Anthropic is DeepSeek v3, which competes with GPT-4o.”
What we tested: He would argue that, despite the large price difference between o1 and R1, the real danger to OpenAI and Anthropic is DeepSeek V3, which competes with GPT-4o.
Assessment: SupportedSaid
“One thing we do know is that OpenAI and Anthropic will now have to either drop the price of accessing their models, and potentially even the cost of their subscriptions.”
What we tested: OpenAI and Anthropic will have to cut API prices, and possibly subscription prices.
What happened: The announcement says the reduction is in effect and applies to the same o3 model. This establishes OpenAI's part of the prediction. (OpenAI, )
Assessment: Can’t be checkedSaid
“Fundamentally, the magic has died. There’s no halo around Sam Altman or Dario Amodei’s head anymore, as their only real argument was “we’re the only ones that can do this,” something that nobody should’ve believed in the first place.”
What we tested: Altman and Amodei have lost their aura because their claim to be uniquely capable has been disproven.
Assessment: Can’t be checkedSaid
“In any case, if I had to guess, the result will be the markets accepting that generative AI isn’t the future.”
What we tested: His guess is that markets will come to accept that generative AI is not the future.
Assessment: UnresolvedSaid
“OpenAI and Anthropic no longer have moats to raise capital with.”
What we tested: OpenAI and Anthropic no longer have moats with which to raise capital.
Assessment: Can’t be checkedSaid
“As I’ve said before, I believe we’re at peak AI, and now that generative AI has been commoditized, the only thing that OpenAI and Anthropic have left is their ability to innovate, which I’m not sure they’re capable of doing.”
What we tested: We are at peak AI, and OpenAI and Anthropic may not be capable of the innovation they now depend on.
Assessment: UnresolvedSaid
“There really isn’t anything special about any of these companies anymore — they have no moat, their infrastructural advantage is moot, and their hordes of talent irrelevant.”
What we tested: LLM companies like OpenAI have no moat; their infrastructure advantage and talent no longer matter.
Assessment: UnresolvedSaid
“Right now, I do not believe that there are companies that are stymied by the pricing that OpenAI and their ilk offer, nor do I think there are many companies or use cases that don’t exist because Large Language Models are too expensive.”
What we tested: Right now he does not believe any companies are stymied by the pricing of OpenAI and similar firms, nor that there are many companies or use cases that don't exist because LLMs are too expensive.
Assessment: Can’t be checkedSaid
“I could absolutely see Meta releasing its own version of DeepSeek’s models — it has the GPUs and Zuckerberg can never be fired, meaning that if he decided to simply throw billions of dollars into specifically creating his own deep-discounted LLMs to wipe out OpenAI he absolutely could.”
What we tested: He could absolutely see Meta releasing its own version of DeepSeek's models, and if Zuckerberg decided to throw billions into deep-discounted LLMs to wipe out OpenAI, he absolutely could.
Assessment: UnresolvedSaid
“As a result, I don’t really see anything changing, beyond the eventual collapse of the API market for companies like Anthropic and OpenAI.”
What we tested: The API market for OpenAI and Anthropic will eventually collapse, with little else changing.
Assessment: ContradictedSaid
“Large Language Models (and reasoning models) are niche.”
What we tested: LLMs and reasoning models are niche products.
What happened: The November survey estimated broad public participation, with search, explanation and writing leading uses. Material qualifications in this same source are possible online-panel selection bias and light usage: its conservative weekly-use estimate was about 29%, and most adults did not use LLMs in a typical week. (Rethink Priorities, )
Assessment: Can’t be checkedSaid
“It’s not essential, and it’s only been treated as such because the media (and the markets) ran away with a narrative they barely understood.”
What we tested: ChatGPT is not essential and has only been treated as essential because the media and markets ran away with a narrative they barely understood.
Assessment: UnresolvedSaid
“In any case, the massive expense of running generative models hasn’t been the limiter on their deployment or success — you can blame that on the fact that they, as a piece of technology, are neither artificial intelligence nor capable of providing the kind of meaningful outcomes that would make them the next Smartphone.”
What we tested: The massive expense of running generative models hasn't been what limits their deployment or success; the limit is that they are neither artificial intelligence nor capable of the meaningful outcomes that would make them the next smartphone.
Assessment: Can’t be checkedSaid
“It’s all been a con, a painfully-obvious one, one I’ve been screaming about since February 2024, trying to explain that beneath the hype was an industry that provided modest-at-best outcomes rather than resembling any kind of “next big thing.””
What we tested: The generative AI industry has been a con delivering modest-at-best outcomes.
Assessment: UnresolvedSaid
““AGI” isn’t coming.”
What we tested: AGI is not coming.
Held from this post (23)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The record does not meet the current rules. Our reading, not assessed: OpenAI and Anthropic sell unprofitable, unreliable, hallucination-prone products, and new products like Sora are too costly for poor results.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: DeepSeek's models are open source, about 30 times cheaper to run, and can run locally on modest hardware.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: GPT-4o cost about $100 million to train, and o1 likely cost more.
- The criteria are ambiguous. Our reading, not assessed: OpenAI and Anthropic did not make their models more efficient because they did not have to.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: DeepSeek R1 is competitive with OpenAI's o1 while being roughly 96% cheaper to run, and can run locally.
- The two reviews reached different assessments. Our reading, not assessed: Reasoning models and LLMs in general are now commoditized.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The two reviews read the claim differently
- Attribution is uncertain
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: The premise behind the AI bubble, that these models could not be built without burning massive amounts of cash, straining the power grid and blowing past emissions goals, and that these were necessary costs of powerful AI, was obviously untrue.
- The two reviews reached different assessments. Our reading, not assessed: DeepSeek has commoditized the Large Language Model, publishing both the source code and the guide to building your own.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: OpenAI and Anthropic each lose billions of dollars a year after revenue.
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: Generative AI solves no trillion-dollar problems and produces no profitable outcomes for any business.
- The two reviews read the claim differently