Breaking news: Scaling will never get us to AGI
We recorded 6 claims from this post. 6 were checked against evidence. 4 more are held. Read the post at its source.
Claims from this post (6)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: ContradictedSaid
Causal claim
“This is why driverless cars are still just demos, and why LLMs will never be reliable.”
Our reading: Poor generalization beyond training data is why driverless cars are still just demos.
No horizon Evidence as of
Limitation: The decisive operational account is the company's regulatory filing.
Assessment: UnresolvedSaid
Causal claim
“It also drives their greediness with data, and even the ethical choices their developers have been making.”
Our reading: Neural networks' trouble generalizing beyond their training space, at least in the configurations dominant over the last three decades, also drives their greediness with data and even the ethical choices their developers have been making.
No horizon Evidence as of
Limitation: The technical evidence concerns particular recurrent networks and tasks, and its link to data demand is tentative [e1].
Assessment: SupportedSaid
Observation
“Neural networks (at least in the configurations that have been dominant over the last three decades) have trouble generalizing beyond the multidimensional space that surrounds their training examples.”
Our reading: Neural networks in the configurations dominant for the last three decades have trouble generalizing beyond the space surrounding their training examples.
No horizon Evidence as of
Limitation: The outcome depends on reading the spatial language as unfamiliar conditions rather than a precise geometric boundary.
Assessment: SupportedSaid
Observation
“current models aren’t literally look up tables; they can generalize to some degree, but not enough.”
Our reading: Current AI models can generalize to some degree but not enough.
No horizon Evidence as of
Limitation: The outcome depends on interpreting 'not enough' through the surrounding discussion of reliability.
Assessment: UnresolvedSaid
Forecast
“In short, diminishing returns are what we can expect.”
Our reading: Scaling data for current AI models will yield diminishing returns.
No deadline stated Evidence as of
Limitation: A narrower claim about a specified loss curve could be supported by individual experiments.
Assessment: UnresolvedSaid
Forecast
“Sooner or later, the exponential greed for data will exceed what is available.”
Our reading: The exponentially growing demand for training data will eventually exceed the data available.
No deadline stated Evidence as of
Limitation: Epoch's quantitative estimate covers public human-generated text, not all usable training data [e1].
Held from this post (4)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: LLMs will never be reliable, because neural networks (at least in the configurations dominant over the last three decades) have trouble generalizing beyond the space surrounding their training examples.
- The two reviews read the claim differently
- The two reviews read the claim differently