What if Generative AI turned out to be a Dud?
We recorded 23 claims from this post. 6 were checked against evidence. 17 can’t be checked. 6 more are held. Read the post at its source.
Claims from this post (23)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
“With the possible exception of the quick to rise and quick to fall alleged room-temperature superconductor LK-99, few things I have ever seen have been more hyped than generative AI.”
What we tested: Generative AI is among the most hyped things the author has ever seen.
Assessment: Can’t be checkedSaid
“Those revenues genuinely could grow by 1000x, but that’s mighty speculative. We shouldn’t simply assume it.”
What we tested: We should not assume generative AI revenues will grow 1000x, which is highly speculative.
Assessment: Can’t be checkedSaid
“But, to begin with, the revenue isn’t there yet, and might never come.”
What we tested: Generative AI revenue is not yet there and might never materialize.
Assessment: UnresolvedSaid
“I think coders will remain happy with generative AI assistance; its autocomplete nature is fabulous for their line of work, and they have the training to detect and fix the not-infrequent errors.”
What we tested: Marcus thinks coders will remain happy with generative AI assistance, because its autocomplete nature is fabulous for their work and they have the training to detect and fix its not-infrequent errors.
Assessment: Can’t be checkedSaid
“Other potential paying customers may lose heart quickly.”
What we tested: Other potential paying customers of generative AI, beyond coders and undergraduates, may lose heart quickly.
Assessment: Can’t be checkedSaid
“But neither coding nor high-speed, mediocre quality copy-writing are remotely enough to maintain current valuation dreams.”
What we tested: Coding and mediocre copywriting use cases are nowhere near enough to justify current generative AI valuations.
Assessment: Can’t be checkedSaid
“If Evans’ experience is a canary in a coal mine, the whole generative AI field, at least at current valuations, could come to a fairly swift end.”
What we tested: If user disillusionment like Evans' is representative, the generative AI field at current valuations could end fairly swiftly.
Assessment: Can’t be checkedSaid
“Even OpenAI could have a hard time following through on its $29 billion valuation; competing startups valued in the low billions might well eventually collapse, if year after year they manage only tens or hundreds of millions in revenue.”
What we tested: OpenAI could have difficulty living up to its $29 billion valuation.
Assessment: Can’t be checkedSaid
“Even OpenAI could have a hard time following through on its $29 billion valuation; competing startups valued in the low billions might well eventually collapse, if year after year they manage only tens or hundreds of millions in revenue.”
What we tested: Generative AI startups valued in the low billions might well eventually collapse if they earn only tens or hundreds of millions annually.
Assessment: Can’t be checkedSaid
“Microsoft, up for the year by nearly half, perhaps largely on the promise of generative AI, might see a stock slump; NVIDIA skyrocketing even more, might also fall.”
What we tested: Microsoft's stock, up nearly half for the year perhaps largely on the promise of generative AI, might slump, and NVIDIA's stock, which has risen even more, might also fall.
Assessment: UnresolvedSaid
“Perhaps the only use case that really seemed compelling to either of us economically was search (e.g., using Bing powered by ChatGPT instead of Google Search) — but the technical problems there are immense; there is no reason to think that the hallucination problem will be solved soon.”
What we tested: There is no reason to think that the hallucination problem, part of the immense technical problems facing the search use case, will be solved soon.
Assessment: Can’t be checkedSaid
“If it isn’t, the bubble could easily burst.”
What we tested: If hallucination is not solved soon, the generative AI bubble could easily burst.
Assessment: Can’t be checkedSaid
“We might not get the consumer protections we need, because we are trying to foster something that may not grow as expected.”
What we tested: The US might fail to get needed consumer protections because it is fostering generative AI, which may not grow as expected.
Assessment: Can’t be checkedSaid
“if the premise that generative AI is going to be bigger than fire and electricity turns out to be mistaken, or at least doesn’t bear out in the next decade, it’s certainly possible that we could wind up with what in hindsight is a lot of needless extra tension with China, possibly even a war in Taiwan, over a mirage”
What we tested: If the premise that generative AI will be bigger than fire and electricity proves mistaken, or at least does not bear out in the next decade, it is certainly possible that we could end up with a lot of needless extra tension with China over a mirage, possibly even a war in Taiwan.
Assessment: Can’t be checkedSaid
“Everybody in industry would probably like you to believe that AGI is imminent. It stokes their narrative of inevitability, and it drives their stock prices and startup valuations.”
What we tested: Everybody in industry would probably like you to believe that AGI is imminent, because that belief stokes their narrative of inevitability and drives their stock prices and startup valuations.
Assessment: SupportedSaid
“We have not one, but many, serious, unsolved problems at the core of generative AI — ranging from their tendency to confabulate (hallucinate) false information, to their inability to reliably interface with external tools like Wolfram Alpha, to the instability from month to month (which makes them poor candidates for engineering use in larger systems).”
What we tested: Generative AI has many serious unsolved core problems, including hallucination, unreliable tool use, and month-to-month instability.
What happened: OpenAI explicitly acknowledged continuing factual errors in its released model. The same section reports improvements, so persistence should not be confused with lack of progress. (OpenAI, )
Assessment: SupportedSaid
““Scaling” systems by making them larger has helped in some ways, but not others; we still really cannot guarantee that any given system will be honest, harmless, or helpful, rather than sycophantic, dishonest, toxic or biased.”
What we tested: Scaling has helped only partially; no system can yet be guaranteed honest, harmless, or helpful.
What happened: The study documents tasks where increasing scale worsened performance despite broader gains in prediction and other tasks. It also reports exceptions and trend reversals, so it does not establish that scaling generally fails. (Ian R. McKenzie and colleagues, arXiv, )
Assessment: UnresolvedSaid
“And, reality check, we have no concrete reason, other than sheer technoptimism, for thinking that solutions to any of these problems is imminent.”
What we tested: There is no concrete reason, other than sheer techno-optimism, for thinking that solutions to any of generative AI's core problems (such as hallucination, unreliable tool use and month-to-month instability) are imminent.
Assessment: Can’t be checkedSaid
“It’s foolish to imagine that such challenging problems will all suddenly be solved.”
What we tested: It is foolish to imagine that generative AI's challenging core problems will all suddenly be solved.
Assessment: SupportedSaid
“The technology we have now is built on autocompletion, not factuality.”
What we tested: Current generative AI is built on autocompletion rather than factuality.
What happened: The report explicitly identifies predictive completion as GPT-4’s base objective. It separately reports factuality gains from post-training, supporting the foundation reading while qualifying a literal denial of factuality work. (OpenAI, arXiv, )
Assessment: Can’t be checkedSaid
“If hallucinations aren’t fixable, generative AI probably isn’t going to make a trillion dollars a year. And if it probably isn’t going to make a trillion dollars a year, it probably isn’t going to have the impact people seem to be expecting.”
What we tested: If hallucinations aren't fixable, generative AI probably won't earn a trillion dollars a year or have the expected impact.
Assessment: Can’t be checkedSaid
“And if it isn’t going to have that impact, maybe we should not be building our world around the premise that it is.”
What we tested: If generative AI isn't going to have the impact people seem to expect, maybe we should not be building our world around the premise that it will.
Assessment: Can’t be checkedSaid
“I seriously doubt it. We have not one, but many, serious, unsolved problems at the core of generative AI”
What we tested: Marcus seriously doubts the projections of near-term AGI, including Dario Amodei's projection that we will have AGI in 2-3 years from August 2023.
Held from this post (6)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: AGI is unlikely to arrive in the near term, e.g. within 2-3 years as Dario Amodei projected.
- The criteria are ambiguous. Our reading, not assessed: Generative AI valuations assume trillion-dollar markets while current revenues are reportedly only in the hundreds of millions.
- The criteria are ambiguous. Our reading, not assessed: Undergraduates will continue to use generative AI, but their pockets aren't deep and they will most likely turn to open-source competitors.
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: In Marcus's view, almost everyone is making the fundamental error of believing that generative AI is tantamount to AGI (general-purpose AI as smart and resourceful as humans, if not more so).