Does AI really need a paradigm shift?
We recorded 14 claims from this post. 6 were checked against evidence. 8 can’t be checked. 4 more are held. Read the post at its source.
Claims from this post (14)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“I say that we have given the scaling hypothesis a really good look (with a larger budget than all but a handful of projects in history), and as such the failures of large scale systems are evidence (not proof) that we ought to seriously consider alternatives, e.g., here:”
Our reading: The failures of large-scale systems, after a heavily funded test of the scaling hypothesis, are evidence that the field should seriously consider alternatives to scaling.
No horizon
Qualifier: given failures of large scale systems
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“GPT-2 is both a triumph for empiricism, and, in light of the massive resources of data and computation that have been poured into them, a clear sign that it is time to consider investing in different approaches.”
Our reading: GPT-2 is a clear sign that it is time to consider investing in approaches other than scaling.
No horizon
Qualifier: in light of massive data and compute poured into GPT-2
Assessment: SupportedSaid
Observation
“so many of the unfortunate consequences of a webscraping/big data approach to AI are disproportionately borne by women and minorities.”
Our reading: Many of the harmful consequences of the webscraping/big-data approach to AI fall disproportionately on women and minorities.
No horizon Evidence as of
Limitation: Support depends on reading many qualitatively and including representational and data-exclusion harms.
Assessment: Can’t be checkedSaid
Forecast
“Personally I’d put the probability that we need to shift at 90%, well above the 60% that Alexander suggests, and I would put the probability that we will need to embrace symbol-manipulation as part of the mix at 80%, more than double Alexander’s 34%.”
Our reading: Marcus puts a 90% probability on AI needing a paradigm shift to reach AGI.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“Personally I’d put the probability that we need to shift at 90%, well above the 60% that Alexander suggests, and I would put the probability that we will need to embrace symbol-manipulation as part of the mix at 80%, more than double Alexander’s 34%.”
Our reading: Marcus puts an 80% probability on AGI needing symbol manipulation as part of the mix.
No horizon
Assessment: SupportedSaid
Observation
“The basic finding was: loads of things scale, but not at all ( as foreseen both by Kurenkov in his essay last year and in my much lampooned but basically accurate essay, Deep learning is hitting a wall).”
Our reading: BIG-bench shows many capabilities improve with scale but not all, which bears out Marcus's 'Deep learning is hitting a wall' essay.
No horizon Evidence as of
Limitation: The finding does not establish a permanent scaling barrier or what happens beyond the tested model sizes [e1].
Assessment: SupportedSaid
Observation
“The massive paper looks at scaling on many measures, and sees progress on some– but not others. The stuff that isn’t scaling is the “wall”.”
Our reading: The BIG-bench paper looked at scaling on many measures and found progress on some but not others, and the measures that did not scale in it are what Marcus calls the 'wall'.
No horizon Evidence as of
Limitation: The finding applies to the evaluated models, scales and metrics.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“That’s why we need new approaches, exactly as I have been arguing.”
Our reading: AI research needs new approaches rather than scale alone.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“While there’s no denying that the two of us have often clashed, on this point—the point about scale alone not likely to be enough, and about the need for some new discovery (i.e., paradigm shifts)—we are actually in complete agreement.”
Our reading: Scale alone is not likely to be enough, and new discoveries (paradigm shifts) are needed, a point on which Marcus and LeCun agree.
No horizon
Assessment: UnresolvedSaid
Conditional forecast
“👉explicit cognitive models will be crucial”
Our reading: Whenever AGI arrives, explicit cognitive models will be crucial to it.
No deadline stated Evidence as of
Only if: whenever AGI comes
Assessment: UnresolvedSaid
Conditional forecast
“👉operations over variables (including storing, retrieving and comparing values) will be crucial”
Our reading: Whenever AGI arrives, operations over variables, including storing, retrieving and comparing values, will be crucial to it.
No deadline stated Evidence as of
Only if: whenever AGI comes
Assessment: UnresolvedSaid
Conditional forecast
“👉an explicit type/token distinction will be crucial”
Our reading: Whenever AGI arrives, an explicit type/token distinction will be crucial to it.
No deadline stated Evidence as of
Only if: whenever AGI comes
Assessment: Can’t be checkedSaid
Forecast
“That said, I, of course, could be right about foreseeing the need for a paradigm shift, but wrong about what that paradigm shift turns out to be.”
Our reading: A paradigm shift may be needed, but it could take a different form than the neurosymbolic one Marcus advocates.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“In my view, no way; I think the actual odds are less than 20%.”
Our reading: The odds that scaling plus CNNs, RNNs and Transformers alone reach AGI with reliable reasoning are below 20%.
No horizon
Qualifier: with scaling and CNNs + RNNs + Transformers alone
Held from this post (4)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: There is at least a little more headroom left for large language models, though the real controversy is whether that is enough.
- The two reviews read the claim differently