AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
We recorded 17 claims from this post. 14 were checked against evidence. 3 can’t be checked. 11 more are held. Read the post at its source.
Claims from this post (17)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
Forecast
“The neural network approach known as generative AI — especially its poster chatbot ChatGPT – has captured the world’s imagination, but will soon flame out.”
Our reading: Generative AI, especially ChatGPT, will soon flame out.
No deadline stated Evidence as of
Limitation: OpenAI's disclosure is self-reported.
Assessment: ContradictedSaid
Forecast
“I fully expect that the generative AI bubble will begin to burst within the next 12 months, for many reasons:”
Our reading: The generative AI bubble will begin to burst by July 28, 2025.
Deadline passed Evidence as of
Limitation: The result depends on interpreting burst as a broad, sustained unwinding.
Assessment: ContradictedSaid
Observation
“The current approach has reached a plateau”
Our reading: The current generative AI approach has reached a plateau.
No horizon Evidence as of
Limitation: Arena preferences can reflect presentation as well as correctness.
Assessment: UnresolvedSaid
Observation
“There is no killer app”
Our reading: Generative AI has no killer app.
No horizon Evidence as of
Limitation: Killer app remains a qualitative threshold.
Assessment: UnresolvedSaid
Observation
“Nobody has a moat”
Our reading: No generative AI company has a competitive moat.
No horizon Evidence as of
Limitation: The universal claim requires unusually broad evidence to support.
Assessment: UnresolvedSaid
Observation
“People are starting to realize all of the above.”
Our reading: People are starting to realize generative AI has plateaued, lacks a killer app, still hallucinates and errs, and has no moat.
No horizon Evidence as of
Limitation: The population and size of the claimed change are unspecified.
Assessment: Can’t be checkedSaid
Conditional forecast
“I think that GenAI is vastly overrated and overhyped, but fear that its collapse may well lead to an AI winter of sorts, like what happened in the mid-1980s, when AI “expert systems” rapidly ascended and rapidly fell.”
Our reading: GenAI is vastly overrated, and its collapse may well lead to an AI winter of sorts.
No horizon
Only if: if GenAI collapses
Assessment: UnresolvedSaid
Conditional forecast
“That said, I am certain that the impending collapse won’t lead to the absolute death of AI. There is too much at stake.”
Our reading: The impending generative AI collapse will not lead to the absolute death of AI.
No deadline stated Evidence as of
Only if: after the generative AI collapse
Assessment: UnresolvedSaid
Forecast
“The biggest intrinsic failings of generative AI have to do with reliability, in a way that I believe can never be solved, given their inherent nature.”
Our reading: Generative AI's reliability failings can never be solved, given its inherent nature.
No deadline stated Evidence as of
Limitation: The cited empirical study covers two benchmarks and a finite set of agents.
Assessment: SupportedSaid
Observation
“Consequently, any “reasoning” or “planning” in LLMs is hit or miss, viable if the particulars of a given situation are close enough to what is in the training set, but highly vulnerable if they are not.”
Our reading: Marcus says any reasoning or planning in LLMs is hit or miss: viable if a situation's particulars are close enough to the training set, but highly vulnerable if they are not.
No horizon Evidence as of
Limitation: Reasoning models released after July 2024 are not assessed.
Assessment: SupportedSaid
Observation
“Hallucinations and boneheaded errors of reasoning and what I like to call discomprehensions are, as far as I can tell, an inevitable byproduct of LLMs; certainly they have yet to disappear, despite many promises to the contrary.”
Our reading: Marcus says that, as far as he can tell, hallucinations, boneheaded reasoning errors and 'discomprehensions' are an inevitable byproduct of LLMs, and that they certainly have not yet disappeared despite many promises to the contrary.
No horizon Evidence as of
Limitation: Only the 'yet to disappear' part is scored.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Since LLMs inevitably hallucinate and are constitutionally incapable of checking their own work, there are really only two possibilities: we abandon them, or we use them as components in larger systems that can reason and plan better”
Our reading: LLMs should either be abandoned or used only as components within larger systems that reason and plan better.
No horizon
Qualifier: since LLMs inevitably hallucinate
Assessment: Can’t be checkedSaid
Observation
“The current, short-term investment climate is thus hugely biased against genuine novelty and outside the box thinking.”
Our reading: The current AI investment climate is hugely biased against genuine novelty.
No horizon Evidence as of
Limitation: The Crunchbase figure covers all of 2024 and was published after the post. The claim is about the climate in July 2024.
Assessment: UnresolvedSaid
Observation
“So for the most part, we’ve been stuck in a world in which all the major tech companies are building exactly the same thing (bigger and bigger LLMs based on more data), achieving almost exactly the same results (a passel of GPT-4 level models, hardly different from each other, all struggling with hallucinations and boneheaded errors), and hardly any investment in anything else.”
Our reading: Marcus says that, for the most part, all the major tech companies are building the same thing (bigger LLMs on more data) and getting almost the same results (GPT-4-level models that barely differ and all struggle with hallucinations and boneheaded errors), with hardly any investment in anything else.
No horizon Evidence as of
Limitation: Company evaluations are not a substitute for consistent independent comparisons.
Assessment: SupportedSaid
Observation
“Much as I admire the new pair of systems, they do have a serious shortcoming and one that is not unfamiliar. The systems both rely on a kind of cheat on the input: trained human coders translate all the Olympiad's input sentences into mathematical form.”
Our reading: AlphaProof and AlphaGeometry 2 rely on human coders translating Olympiad problems into formal form, a serious shortcoming.
No horizon Evidence as of
Limitation: The assessment concerns the July 2024 demonstration, not later systems.
Assessment: SupportedSaid
Observation
“Nonetheless, GDM's new work is powerful proof of concept (pun intended!). Neural networks and symbol systems can be profitably combined, contra Hinton and LeCun.”
Our reading: Google DeepMind's work shows neural networks and symbolic systems can be profitably combined.
No horizon Evidence as of
Limitation: Technical usefulness does not imply financial profitability.
Assessment: UnresolvedSaid
Forecast
“There can be no path to AGI without neurosymbolic AI.”
Our reading: AGI cannot be achieved without neurosymbolic AI.
No deadline stated Evidence as of
Limitation: Public demonstrations may not disclose enough implementation detail to classify a system's methods.
Held from this post (11)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: LLMs cannot sanity-check their own work.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: Generative AI systems still make boneheaded errors.
- The criteria are ambiguous. Our reading, not assessed: Marcus says the lack of sanity-checking leads LLMs to flub arithmetic, make boneheaded errors, make stuff up, occasionally defame people and so on, over and over again, in literally every transformer released from GPT-2 to SearchGPT.
- The criteria are ambiguous. Our reading, not assessed: Marcus says LLMs literally do not reason through problems, and that in this riddle case ChatGPT instead retrieved an answer to a slightly different problem, which was wrong.
- The two reviews reached different assessments. Our reading, not assessed: Marcus says OpenAI has for the most part stuck to Hinton and LeCun's dogma, putting the far greater part of its efforts into scaling LLMs and trying hard to eschew symbols and symbolic rules, hiding them behind the curtain when they are used.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: Marcus says one finds over and over that LLMs are better at things they have encountered before than at things they haven't, and are especially poor when new items differ in important and subtle ways from old ones.
- The record does not meet the current rules. Our reading, not assessed: Generative AI may never again be as popular as in the past year, but new, better techniques addressing its failings will come.
- The record does not meet the current rules. Our reading, not assessed: Generative AI systems still hallucinate.