Superhuman AGI is not nigh
We recorded 14 claims from this post. 2 were checked against evidence. 12 can’t be checked. 5 more are held. Read the post at its source.
Claims from this post (14)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
“Skeptical of this, I offered Musk a $1 million bet that AGI superior to the smartest individual humans would not arrive by the end of 2025, reprinted here for those who missed it.”
What we tested: AGI superior to the smartest individual humans will not arrive by the end of 2025.
What happened: The retrospective report says every ARC-AGI-2 task was solved by at least two independent non-expert human testers. Its analysis identifies continuing generalization limitations in 2025 AI systems. (ARC Prize Foundation, via arXiv, )
Assessment: Can’t be checkedSaid
“Right now there are likely hundreds, maybe thousands or even millions of such things.”
What we tested: As of April 2024 there are likely hundreds or more pieces of intellectual work that the best humans can do and AI cannot.
Assessment: Can’t be checkedSaid
“Reliably drive a car in a novel location that they haven’t previously encountered, even in the face of unusual circumstances like hand-lettered signs, without the assistance of other humans.”
What we tested: AI is unlikely by end of 2025 to reliably drive a car in novel locations with unusual circumstances without human assistance.
Assessment: Can’t be checkedSaid
“Watch previously unseen mainstream movie and be able to follow plot twists and know when to laugh, and be able to summarize it without giving away any spoilers or making up anything that didn’t actually happen”
What we tested: AI is unlikely by end of 2025 to watch an unseen mainstream movie, follow plot twists, know when to laugh, and summarize it without spoilers or fabrications.
Assessment: Can’t be checkedSaid
“Drive off-road vehicles, without maps, across streams, around obstacles such as fallen trees, and so on.”
What we tested: AI is unlikely by end of 2025 to drive off-road vehicles without maps across streams and around obstacles.
Assessment: Can’t be checkedSaid
“Learn to ride a mountain bike off-road through forest trails”
What we tested: AI is unlikely by end of 2025 to learn to ride a mountain bike off-road through forest trails.
Assessment: Can’t be checkedSaid
“Clean unfamiliar houses and apartments”
What we tested: AI is unlikely by end of 2025 to clean unfamiliar houses and apartments.
Assessment: Can’t be checkedSaid
“Walk into random kitchens and prepare a cup of coffee or make breakfast”
What we tested: AI is unlikely by end of 2025 to walk into random kitchens and prepare coffee or breakfast.
Assessment: Can’t be checkedSaid
“Babysit children in an unfamiliar homes and keep them safe”
What we tested: AI is unlikely by end of 2025 to babysit children in unfamiliar homes and keep them safe.
Assessment: Can’t be checkedSaid
“Tend to the physical and psychological needs of an elderly or infirm person”
What we tested: AI is unlikely by end of 2025 to tend to the physical and psychological needs of an elderly or infirm person.
Assessment: Can’t be checkedSaid
“Write cogent, persuasive legal briefs without hallucinating any cases”
What we tested: AI is unlikely by end of 2025 to write cogent, persuasive legal briefs without hallucinating any cases.
Assessment: Can’t be checkedSaid
“None seem to me to be within immediate reach.”
What we tested: None of the listed tasks seem within immediate reach of AI.
Assessment: UnresolvedSaid
“I couldn’t agree more. And parallel arguments could be made in many domains.”
What we tested: Marcus endorses the view that ChatGPT and similar tools will not themselves be virtual scientists anytime soon, and says parallel arguments could be made in many domains.
Assessment: Can’t be checkedSaid
“Learn and master the basics of almost any new video game within a few minutes or hours, and solve original puzzles in the alternate world of that video game.”
What we tested: Marcus seriously doubts that, by the end of 2025, AI will be able to learn and master the basics of almost any new video game within a few minutes or hours and solve original puzzles in the alternate world of that video game.
Held from this post (5)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The record does not meet the current rules. Our reading, not assessed: Marcus seriously doubts that AI will be able, by the end of 2025, to write engaging biographies and obituaries without obvious hallucinations that aren't grounded in reliable sources.
- The record does not meet the current rules. Our reading, not assessed: If Marcus understands it correctly, Musk's claim that AI will be able to do any intellectual work any human can do by the end of next year is almost certain to be untrue.
- Attribution is uncertain
- The record does not meet the current rules. Our reading, not assessed: Marcus seriously doubts that AI will be able, by the end of 2025, to learn and master the basics of almost any new video game within a few minutes or hours and solve original puzzles in that game's world.
- The criteria are ambiguous. Our reading, not assessed: As of April 2024, all listed tasks (including Pulitzer-caliber books, Oscar-caliber screenplays, Nobel-caliber discoveries) appear outside the bounds of current AI.