“Without external aids it won’t be able beat Meta’s Cicero in Diplomacy; it won’t be able to drive a car reliably; it won’t be able to reliably guide a robot like Optimus to be anything like as versatile as Rosie the Robot.”
Our reading, the sentence that was checked:
Without external aids, GPT-5 will not be able to beat Meta's Cicero at Diplomacy, will not be able to drive a car reliably, and will not be able to reliably guide a robot like Optimus to be anything like as versatile as Rosie the Robot.
Assessment: Unresolved No deadline stated on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, What to Expect When You’re Expecting … GPT-5, paragraph 7.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-what-to-expect-when-youre-expecting-62e-177f3a-a1
We recorded 15 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-07-27. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No deadline stated
- Deadline state on 2026-10-29
- No deadline stated (the release’s reference date)
Conditions
- without external aids (qualifier, stated by the speaker)
From the speaker
- Without external aids
- won’t be able beat Meta’s Cicero in Diplomacy
- won’t be able to drive a car reliably
- won’t be able to reliably guide a robot like Optimus
- anything like as versatile as Rosie the Robot
Our interpretation
- This is a conjunction of three capability predictions about the original GPT-5.
- ‘Without external aids’ restricts the system being tested; it is not a separate future event that must occur.
- Necessary sensors and action interfaces are allowed, but specialist reasoning, planning or driving systems supplying the missing intelligence are excluded.
- Beating Cicero means reproducible competitive success, not one lucky game. Reliable driving means sustained safe performance across representative driving conditions.
- Rosie is read as broad household versatility. The fictional comparison supplies no measurable task list or reliability threshold.
Counts as supported if: Representative controlled evaluations establish the predicted limitations in all three domains under the unaided configuration.
Counts as contradicted if: A reproducible demonstration establishes at least one of the denied capabilities under the unaided configuration. The other components must be assessed separately.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No deadline stated
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The robot study supplies relevant evidence of limitations, but it does not settle all three predictions or precisely match the unaided condition. The searches did not locate matching Cicero or driving evaluations. Missing demonstrations cannot establish inability, and Marcus gave no deadline.
Limitations
- This compound claim cannot be supported from evidence about robotics alone.
- The reliability and versatility thresholds require interpretation.
Evidence (1 item)
Context only: Callum Sharrock and colleagues, arXiv,
all models struggled with different things.
Why it matters here: Butter-Bench includes GPT-5 and documents difficulties controlling a robot on household delivery tasks. Its robot interface and narrow task set do not establish the entire unaided robotics prediction, and it does not test driving or Cicero.
- Publisher
- Callum Sharrock and colleagues, arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Controlled tests of the original GPT-5 against Cicero, on representative driving conditions, and on varied household robot tasks, with all external assistance disclosed. A qualifying success could settle a component early.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, What to Expect When You’re Expecting … GPT-5
Without external aids it won’t be able beat Meta’s Cicero in Diplomacy; it won’t be able to drive a car reliably; it won’t be able to reliably guide a robot like Optimus to be anything like as versatile as Rosie the Robot.
- Where
- paragraph 7
- Published
- 2025-07-27
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-what-to-expect-when-youre-expecting-62e-177f3a-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-what-to-expect-when-youre-expecting-62e-177f3a-a1.