“And, no, it wasn’t about common sense at all.”
Our reading, the sentence that was checked:
Imagen's failure on 'a horse riding an astronaut' was not due to common sense.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Horse rides astronaut, paragraph 65.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-horse-rides-astronaut-6501e4-a2
We recorded 10 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-05-28. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Causal claim
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- it wasn’t about common sense at all
- All good guesses, all wrong
Our interpretation
- The claim concerns the cause of original Imagen’s May 2022 failures on the horse-riding-astronaut prompt.
- In context, “common sense” means a plausibility judgment or learned expectation that makes the model favor ordinary riding arrangements.
- Successful rewording can rule out an absolute inability or unconditional refusal. It does not by itself rule out a contributing plausibility preference.
- The categorical phrase “at all” requires stronger evidence than showing that wording affects the result.
Counts as supported if: Controlled comparisons or model interventions show that plausibility-based judgment did not cause the original failure, while identifying a sufficient alternative mechanism.
Counts as contradicted if: Controlled evidence shows that a plausibility-based judgment or expectation causally contributed to the role reversal.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Does the evidence support the cause?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
On May 25, 2022, Neyshabur reported that adding “on back of” or “on shoulders of” improved Imagen’s chance of depicting a horse riding an astronaut [e5]. The demonstration does not separate wording effects from learned plausibility preferences, so it cannot establish whether those preferences contributed to the original failure [e5]. The claim that common sense played no causal role remains unresolved [e5].
Limitations
- The demonstration provides no repeated-sample success rates or controlled comparison separating wording from scene plausibility [e5].
- Successful rewording establishes neither the presence nor the absence of a plausibility-based contribution to the original failure [e5].
Evidence (1 item)
Context only: Behnam Neyshabur,
helps increase the chance of getting it right!
Why it matters here: The researcher reports improved success after specifying the riding position. This challenges an unconditional refusal explanation, but does not isolate or exclude a contributing preference for plausible scenes.
- Publisher
- Behnam Neyshabur
- Kind of source
- Demonstration
- Describes events of
- Public from
What would settle it
Controlled tests on original Imagen, using repeated samples and matched prompt variants, that separate wording from plausibility effects. These would need to establish a causal contribution from plausibility preferences, or exclude that contribution while identifying a sufficient alternative mechanism.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Horse rides astronaut
And, no, it wasn’t about common sense at all.
- Where
- paragraph 65
- Published
- 2022-05-28
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-horse-rides-astronaut-6501e4-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.