Satya Nadella and the three stages of scientific truth
We recorded 6 claims from this post. 2 were checked against evidence. 4 can’t be checked. 1 more is held. Read the post at its source.
Claims from this post (6)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
“The point was that we probably needed new architectures.”
What we tested: Marcus restates the point of his 2022 essay: AI probably needed new architectures beyond scaling LLMs.
Assessment: Can’t be checkedSaid
“In recent weeks, the third stage has finally arrived.”
What we tested: In recent weeks, as of November 2024, Marcus's 2022 conjecture has reached the stage of being regarded as self-evident, with Microsoft CEO Satya Nadella's remarks as an example. The conjecture is that LLM scaling laws are empirical observations that may not hold and that new approaches would be needed.
Assessment: SupportedSaid
“Many other Tech CEOs and others, including Marc Andreeseen (A16Z), Alexandr Wang (ScaleAI), have also used the phrase “hitting a wall” and the idea of diminishing returns in the last couple weeks.”
What we tested: In the last couple of weeks before November 23, 2024, many other tech CEOs and others, including Marc Andreessen and Alexandr Wang, used the phrase "hitting a wall" and the idea of diminishing returns.
What happened: Wang publicly used the 'wall' framing on Nov 20, 2024, three days before the post. He partly agreed that a wall had been hit, and the report links it to diminishing returns from training. (The Verge, )
Assessment: Can’t be checkedSaid
“What’s at stake? Science gets delayed.”
What we tested: Resistance to criticism of scaling slowed scientific progress in AI.
Assessment: Can’t be checkedSaid
“Almost a half decade lost, when we could have been seriously investing in other approaches and might have made genuine progress towards robust, trustworthy artificial intelligence.”
What we tested: Focusing on LLMs cost AI research almost half a decade. Investing in alternatives might have produced real progress toward robust, trustworthy AI.
Assessment: UnresolvedSaid
“Hundreds of billions of dollars were invested on the assumption that I was wrong, largely to the exclusion of investment in other approaches.”
What we tested: Hundreds of billions of dollars went into LLM scaling on the assumption that scaling would keep working, and alternative approaches got little investment.
Held from this post (1)
These records did not pass every check, so nothing about them is published. Held is not a finding.