Why I don’t share Sam Altman’s confidence that AGI is basically a solved problem
We recorded 16 claims from this post. 8 were checked against evidence. 8 can’t be checked. 3 more are held. Read the post at its source.
Claims from this post (16)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Observation
“I do not share his confidence.”
Our reading: Marcus does not share Sam Altman's confidence that OpenAI knows how to build AGI.
No horizon
Assessment: Can’t be checkedSaid
Observation
“Here is a thread of links to a several recent observations that I think show how far we are from robust intelligence.”
Our reading: Marcus thinks several recent observations show how far AI is from robust intelligence.
No horizon
Qualifier: I think
Assessment: UnresolvedSaid
Observation
“There is no *principled* solution thus far to any of them.”
Our reading: None of the known problems discussed in Marcus's thread has a principled solution thus far.
No horizon Evidence as of
Limitation: This finding does not dispute the narrower observations of continuing reliability failures.
Assessment: SupportedSaid
Observation
“The problem of distribution shift, well-tested in Apple’s 2024 reasoning paper, which goes back to my own 1998 work, continues.”
Our reading: The problem of distribution shift in neural models continues to be unsolved.
No horizon Evidence as of
Limitation: Support concerns continued empirical failures, not a proof that neural models cannot generalize.
Assessment: SupportedSaid
Observation
“LLMs, like their predecessors generalize well to similar items but struggle with reliability in unfamiliar regimes, even minor variations sometimes break them.”
Our reading: LLMs generalize well to similar items but are unreliable in unfamiliar regimes, and minor variations sometimes break them.
No horizon Evidence as of
Limitation: The cited evidence concerns mathematical word problems, not every application of LLMs.
Assessment: UnresolvedSaid
Observation
“A recent paper on Putnam math problems showed a similar failure on distribution shift, showing 30% decrement by o1 on math problems with minor variations eg on variable names.”
Our reading: A recent paper showed o1's performance drops about 30% on Putnam math problems with minor variations like changed variable names.
No horizon Evidence as of
Limitation: The evidence does not securely identify and date the manuscript version available before the claim.
Assessment: SupportedSaid
Observation
“Commonsense reasoning is still dodgy, as it has long been, as Ernie Davis and I just reviewed here..”
Our reading: Commonsense reasoning in current AI systems remains unreliable, as it long has been.
No horizon Evidence as of
Limitation: Commonsense is broader than the tasks tested.
Assessment: Can’t be checkedSaid
Observation
“o1 probably isn’t as robust as people might have hoped. Even on certain benchmarks, results may be fragile or difficult to replicate:”
Our reading: o1 is probably less robust than hoped, and some benchmark results may be fragile or hard to replicate.
No horizon
Assessment: Can’t be checkedSaid
Observation
“In domains where that’s not possible, o1 may not even be that much better than GPT-4 (e.g on some language tasks), as you can see if you read OpenAI’s reports carefully.”
Our reading: Where data augmentation isn't possible, o1 may not be much better than GPT-4, for example on some language tasks.
No horizon
Qualifier: in domains where data augmentation isn't possible
Assessment: Can’t be checkedSaid
Observation
“In o1’s best performances cases, there may have been heavy data augmentation that is not possible in more open-ended domains.”
Our reading: In o1's best-performance cases there may have been heavy data augmentation that is not possible in more open-ended domains.
No horizon
Scope: in o1's best-performance cases
Assessment: Can’t be checkedSaid
Forecast
“The presentation of o3 was problematic, and no clean test of out-of-domain generalization was provided, suggesting that we may see the kinds of problems with distribution shift as ever, outside of semi-closed domains where synthetic data can readily be generated.”
Our reading: Marcus says the presentation of o3 was problematic and gave no clean test of out-of-domain generalization, which suggests o3 may show the same distribution-shift problems as ever outside semi-closed domains where synthetic data can readily be generated.
No horizon
Scope: outside semi-closed domains where synthetic data can readily be generated
Assessment: SupportedSaid
Observation
“Many leading figures in the field have acknowledged that we may have reached a period of diminishing returns of pure LLM scaling, much as I anticipated in 2022.”
Our reading: Many leading figures in the field have acknowledged that pure LLM scaling may have reached a period of diminishing returns, much as Marcus anticipated in 2022.
No horizon Evidence as of
Limitation: The quantity 'many' remains interpretive.
Assessment: Can’t be checkedSaid
Forecast
“[Note also that the current idea, test-time “scaling”, is very expensive, and will likely prove unreliable out of domains where augmentation is adequate, in which case it will get comparatively little commercial traction.”
Our reading: Test-time scaling is very expensive and will likely prove unreliable outside domains where data augmentation is adequate.
No horizon
Qualifier: outside domains where augmentation is adequate
Assessment: UnresolvedSaid
Observation
“Even o1 is losing money; o3 isn’t even available, and perhaps cannot be run at a profit.]”
Our reading: OpenAI's o1 is losing money.
No horizon Evidence as of
Limitation: The evidence covers only the ChatGPT Pro tier. It does not cover o1 sold through other channels, such as ChatGPT Plus or the API.
Assessment: UnresolvedSaid
Causal claim
“As I noted in 2001, the lack of distinct, accessible, reliable database-style records leads to hallucinations.”
Our reading: The lack of distinct, accessible, reliable database-style records leads to hallucinations, as Marcus noted in 2001.
No horizon Evidence as of
Limitation: The historical search was not an exhaustive examination of every 2001 publication.
Assessment: Can’t be checkedSaid
Observation
“But so far I have not seen a principled solution to even one of these longstanding problems, and that tells me a lot.”
Our reading: So far Marcus has not seen a principled solution to even one of these longstanding problems, and he says that tells him a lot.
No horizon
Qualifier: so far
Held from this post (3)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The criteria are ambiguous. Our reading, not assessed: If test-time scaling proves unreliable outside domains where augmentation is adequate, it will get comparatively little commercial traction.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: Despite many promises to the contrary, the lack of reliable database-style records and the hallucinations it causes still routinely lead to inaccurate news summaries, defamation, fictional sources, incorrect advice, and unreliability.