Scott Alexander’s Misleading Victory Lap
We recorded 8 claims from this post. 4 were checked against evidence. 4 can’t be checked. 5 more are held. Read the post at its source.
Claims from this post (8)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
Observation
“The field of AI is littered with Band-Aid solutions that work for one problem, and not the next.”
Our reading: The AI field is full of Band-Aid solutions that work for one problem but not the next.
No horizon Evidence as of
Limitation: The result depends on interpreting the metaphor as limited transfer, rather than deliberate engineering of superficial fixes.
Assessment: UnresolvedSaid
Observation
“(GPT 4o also failed to accurately diagnose its own errors, misreading its own output, which represent a different kind of failure of compositionality.)”
Our reading: GPT-4o failed to accurately diagnose errors in its own image outputs.
No horizon Evidence as of
Limitation: The assessment concerns the reported episodes, not a universal inability to inspect images.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“But it is not reasonable to conclude from the occasional success on one sort of publicly-discussed set of prompts that GPT has mastered compositionality writ large.”
Our reading: It is not reasonable to conclude from occasional success on one sort of publicly discussed set of prompts that GPT has mastered compositionality in general.
No horizon
Scope: from occasional success on one sort of publicly-discussed set of prompts
Assessment: Can’t be checkedSaid
Causal claim
“One is data contamination, since any given prompt that has been circulated publicly potentially becomes part of the training data, including with custom data augmentation, especially with large armies of subcontractors.”
Our reading: Publicly circulated prompts may become part of training data (including via data augmentation by subcontractors), making success on them unreliable evidence of general ability.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“It is an enormous, unjustified leap from the motte of “the systems can now solve some compositionality problems” to the bailey of “the systems have mastered compositionality”.”
Our reading: It is an enormous, unjustified leap from 'the systems can now solve some compositionality problems' to 'the systems have mastered compositionality'.
No horizon
Assessment: UnresolvedSaid
Observation
“It took me less than an hour to find half a dozen clear counterexamples in which ChatGPT 4o failed with respect to compositionality, evident even to a teenager.”
Our reading: In less than an hour, Marcus found half a dozen clear counterexamples in which ChatGPT 4o failed with respect to compositionality, failures evident even to a teenager.
No horizon Evidence as of
Limitation: The evidence does not establish Marcus's discovery interval, identify all six claimed prompt-output pairs, or verify their model identity and clarity to non-specialists.
Assessment: Can’t be checkedSaid
Conditional forecast
“When we are really at AGI it will be much harder to do so.”
Our reading: Once AI truly reaches AGI, finding compositionality counterexamples will be much harder than it is now.
No horizon
Only if: when we are really at AGI
Assessment: UnresolvedSaid
Forecast
“I predict that such counterexamples will still be easy to find at the end of 2027 because of limitations inherent in LLMs.”
Our reading: At the end of 2027 it will still be easy to find clear counterexamples where LLM-based generative AI fails at compositionality, because of limitations inherent in LLMs.
Not yet due Evidence as of
Limitation: Existing failures cannot establish that comparable failures will remain easy to find at the deadline.
Held from this post (5)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: GPT-4o succeeds on previously discussed prompts but does worse on new ones, suggesting its compositional ability is far from robust.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: From what Marcus understands, AI performance on Chollet et al.'s ARC-AGI-2, including a section focused on compositionality, is still poor.