Has Sam Altman gone full Gary Marcus?
We recorded 3 claims from this post. 3 can’t be checked. 3 more are held. Read the post at its source.
Claims from this post (3)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
“The hubris of Spring of 2022 has to some small but detectable extent diminishing.”
What we tested: As of November 2023, the hubris of spring 2022 has diminished to some small but detectable extent.
Assessment: Can’t be checkedSaid
“GPT-4 (introduced in the intervening period) is surely better, but people have little by little started to recognize that scoring better on benchmarks is NOT the same as making foundational progress.”
What we tested: GPT-4 is surely better than the models of spring 2022, but people have little by little started to recognize that scoring better on benchmarks is not the same as making foundational progress.
Assessment: Can’t be checkedSaid
“The sooner we stop climbing the hill we are on, and start looking for new paradigms, the better.”
What we tested: AI research should stop pursuing the current LLM/scaling approach as soon as possible and look for new paradigms.
Held from this post (3)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The criteria are ambiguous. Our reading, not assessed: Large language models will not get us to AGI; a new paradigm is needed.
- The two reviews reached different assessments. Our reading, not assessed: As of November 2023, little has fundamentally changed since March 2022: hallucinations remain rampant and LLMs still make many ridiculous errors.