Top 5 reasons OpenAI is probably not worth 90 billion dollars
We recorded 11 claims from this post. 5 were checked against evidence. 6 can’t be checked. 5 more are held. Read the post at its source.
Claims from this post (11)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
“That projection probably comes from their two best quarters, and some data suggest their spectacular growth has already been a slow down after the amazing initial takeoff.”
What we tested: OpenAI's projected $1 billion in revenue this year probably comes from its two best quarters, and some data suggest its growth has already slowed after the initial takeoff.
Assessment: Can’t be checkedSaid
“Current models still have a lot of problems, necessitating future models. Future models will likely be bigger models, even more expensive to train, even more expensive to operate, further reducing profits.”
What we tested: OpenAI's future models will likely be bigger and more expensive to train and operate, further reducing profits.
Assessment: UnresolvedSaid
“If OpenAI were obliged to retrain models only on materials for which they had consent, it significantly would weaken their results, and cost a great deal of money to do that retraining.”
What we tested: If OpenAI had to retrain only on consented materials, its results would be significantly weakened and retraining would be very costly.
Assessment: Can’t be checkedSaid
“In the current state, even small changes to the underlying database might require complete retraining.”
What we tested: Currently, even small changes to an LLM's training data might require complete retraining.
Assessment: UnresolvedSaid
“There’s not a huge moat here to protect OpenAI from competitors. The central technology at OpenAI (as far as anyone knows) is large language models, and many other companies know how to build them.”
What we tested: There is not a huge moat to protect OpenAI from competitors because its central technology, as far as anyone knows, is large language models, which many other companies know how to build.
Assessment: Can’t be checkedSaid
“Students don’t as rule have a lot of money; they may quickly turn to free alternatives.”
What we tested: Students using OpenAI products may quickly switch to free alternatives.
Assessment: SupportedSaid
“The competitors may not (yet) be as good, but they are free, which will suit some customers fine.”
What we tested: Free open-source competitors to OpenAI, though perhaps not yet as good, will suit some customers.
What happened: The case study documents a deployed customer application and reports positive reception and further integration. Together with the release terms, this establishes that a free competing model suited a customer. (Meta, )
Assessment: SupportedSaid
“Hallucinations are a huge, unsolved issue.”
What we tested: LLM hallucinations are a huge, unsolved problem.
What happened: OpenAI identifies continuing factual unreliability and recommends safeguards or avoiding some high-stakes uses. The same section reports substantial improvement over preceding models, which limits any reading that hallucination mitigation had made no progress. (OpenAI, )
Assessment: Can’t be checkedSaid
“It might be OK in domains where 70% correct is OK, but unlikely to be acceptable in mission-critical applications.”
What we tested: LLMs may be acceptable where 70% accuracy suffices but are unlikely to be acceptable for mission-critical applications.
Assessment: SupportedSaid
“Explicit verbal instruction is key in training human employees, and LLMs aren’t very good at following those instructions.”
What we tested: LLMs are not very good at following explicit verbal instructions.
What happened: PaLM 2 S satisfied every checked instruction on 43.07% of prompts under strict scoring and 46.95% under relaxed scoring; individual-instruction accuracy was 55.76% and 59.11%, respectively. These August results show frequent explicit-constraint failures, but this smaller model alone cannot establish performance across all leading LLMs. (arXiv; Jeffrey Zhou and colleagues, )
Assessment: Can’t be checkedSaid
“LLMs will likely never deliver on the promise that they are somehow close to AGI.”
What we tested: LLMs will likely never deliver on the promise of being close to AGI.
Held from this post (5)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The record does not meet the current rules. Our reading, not assessed: A billion dollars in OpenAI revenue is not a billion dollars in profit because its models are extremely expensive to train and fairly expensive to operate.
- The criteria are ambiguous. Our reading, not assessed: OpenAI's profit isn't there.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: GPT cannot be trusted to follow the rules of chess despite those rules being in its training data.
- The record does not meet the current rules. Our reading, not assessed: Big customers are still skittish about LLMs: JP Morgan, Apple, and others ban their use at least partly over concerns about data leaks, and reliability is also a problem.