Casey (Newton) Strikes Out
We recorded 10 claims from this post. 6 were checked against evidence. 4 can’t be checked. 8 more are held. Read the post at its source.
Claims from this post (10)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
Observation
“Newton is also conveniently ignoring an absolutely fundamental distinction, between early period of reliable, rapid progress from 2019-2022 (when training for GPT-4) was completed, and a more recent regime in which by many measures, progress has (reportedly) slowed.”
Our reading: After rapid progress in 2019-2022, LLM progress has by many measures reportedly slowed.
No horizon Evidence as of
Limitation: A narrower assessment asking only whether slowdown reports existed would be supported.
Assessment: UnresolvedSaid
Observation
“Diminishing returns doesn’t mean zero improvement, it means that the “laws” of scaling no longer hold, and that the ROI on adding more data and compute are no longer are predictive in the way that they used to be, hence the need for new approaches.”
Our reading: The diminishing returns he describes in AI do not mean zero improvement; they mean the scaling "laws" no longer hold and the returns on adding more data and compute are no longer predictive in the way they used to be, which is why new approaches are needed.
No horizon Evidence as of
Limitation: Important frontier training results were private.
Assessment: UnresolvedSaid
Observation
“We still don’t have driverless cars in 97% of the world after a decade’s promises; hallucinations haven’t gone away. Home humanoid robots are nowhere in sight.”
Our reading: Driverless cars are absent from 97% of the world, hallucinations persist, and home humanoid robots are nowhere in sight.
No horizon Evidence as of
Limitation: The percentage is not reproducible without a denominator.
Assessment: UnresolvedSaid
Observation
“OpenAI has yet to make a profit, and doesn’t anticipate making one before 2029.”
Our reading: OpenAI has not yet made a profit and does not expect to before 2029.
No horizon Evidence as of
Limitation: The original financial reporting could not be read with the required tool.
Assessment: SupportedSaid
Observation
“Part of the value in skepticism is correctly anticipating road blocks, and suggesting alternative directions (as I have for decades with neurosymbolic AI, which is at last coming into vogue, as noted in Fortune earlier this week).”
Our reading: Neurosymbolic AI is finally coming into vogue.
No horizon Evidence as of
Limitation: The result depends on reading vogue as growing attention.
Assessment: SupportedSaid
Observation
“In 2022 I warned that there would be chatbot related deaths such as suicides. This began to happen in 2023 and continued in 2024.”
Our reading: Chatbot-related deaths such as suicides occurred in 2023 and 2024.
No horizon Evidence as of
Limitation: Causation is not established. The Belgian link rests on the widow's account and chat logs she supplied. The Setzer link rests on lawsuit allegations.
Assessment: Can’t be checkedSaid
Forecast
“A massive investment of hundreds of billions is unlikely to pay off.”
Our reading: The massive investment of hundreds of billions of dollars in AI scaling is unlikely to pay off.
No horizon
Assessment: Can’t be checkedSaid
Causal claim
“I don’t doubt this, when stated so vaguely, but you can’t improve the engineering if you can’t identify the problems and weak points, such that you can address them.”
Our reading: You can't improve the engineering if you can't identify its problems and weak points so that you can address them.
No horizon Evidence as of
Limitation: This does not establish that the maxim is true or false.
Assessment: Can’t be checkedSaid
Forecast
“But ignoring all the evidence that suggests that we won’t be there in the very near term does a disservice to his pretense towards objectivity.”
Our reading: Evidence suggests superintelligence will not be reached in the very near term.
No horizon
Qualifier: per available evidence
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Our best defense against the possible dangers of superintelligence must begin with a realistic appraisal of where we are and how much time we have.”
Our reading: Defending against superintelligence dangers must start with a realistic appraisal of current progress and timelines.
No horizon
Held from this post (8)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The criteria are ambiguous. Our reading, not assessed: All seven of Marcus's March 2024 predictions that generative AI economics would be problematic have held.
- The record does not meet the current rules. Our reading, not assessed: Edge cases remain a serious problem for driverless cars, with no generalizable solution yet.
- The two reviews reached different assessments. Our reading, not assessed: Current gains in generative AI are smaller than the rapid gains of 2020-2022.
- The two reviews read the claim differently
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: For 18 months, AI results have looked incremental rather than exponential; Gemini 2 is only modestly better than Gemini 1.5.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: There is growing evidence that there is not enough data to keep scaling LLMs.