“ChatGPT, by the way, also continues to struggle with the basic facts of life:”
Our reading, the sentence that was checked:
ChatGPT still struggles with basic facts of life in image generation.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, What scaling does and doesn’t buy you: peeling back the hype surrounding Google‘s trendy Nano Banana, paragraph 14.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-what-scaling-does-and-doesnt-buy-c1e720-a2
We recorded 8 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-08-27. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- ChatGPT
- continues to struggle
- the basic facts of life
Our interpretation
- In the surrounding discussion of generated pictures, the ordinary reading is that ChatGPT still makes elementary biological or anatomical errors in images.
- Struggles means a demonstrable recurring difficulty, without implying that every output fails or that performance has not improved.
- The ongoing observation is assessed for August 2025 through publication on August 27. This is the latest contemporaneous observation period; the post specifies no reporting period.
- Relevant tasks include depicting or labeling familiar body parts and following simple anatomical part-count instructions. Failure to draw an intentionally unusual animal would establish an instruction-following problem, not necessarily ignorance of normal biology.
Counts as supported if: Dated, independently inspectable ChatGPT outputs from the target period show repeated elementary anatomical, biological, or body-part errors on clear image-generation prompts.
Counts as contradicted if: Representative contemporaneous testing of the relevant elementary tasks shows reliable success and establishes that the alleged errors were misattributed, misinterpreted, or no longer characteristic of the available system.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Davis's July 8, 2025 report shows a successful four-ear rabbit and improvement over November 2024, alongside remaining image-generation failures [e1]. These July tests do not establish whether ChatGPT repeatedly made elementary anatomical or biological errors during August 1–27, 2025, so the observation remains unresolved [e1].
Limitations
- The admitted evidence contains no tests from August 1–27, 2025.
- The July examples do not establish a representative anatomical-error rate.
- Success or failure at drawing an intentionally unusual animal does not by itself establish knowledge of normal biology.
Evidence (1 item)
Context only: Ernest Davis, New York University,
Note: Correct.
Why it matters here: Davis reports success on the four-ear rabbit task in July. This cautions against treating older failures as persistent without retesting, but it neither establishes nor disproves August difficulties.
- Publisher
- Ernest Davis, New York University
- Kind of source
- Demonstration
- Describes events of
- Public from
What would settle it
Independently inspectable tests from August 1–27, 2025, identifying the ChatGPT image system, dates, exact prompts, outputs, and retries. Repeated elementary anatomical errors would support the claim; representative reliable success and evidence that the alleged errors were misinterpreted, misattributed, or no longer characteristic would contradict it.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, What scaling does and doesn’t buy you: peeling back the hype surrounding Google‘s trendy Nano Banana
ChatGPT, by the way, also continues to struggle with the basic facts of life:
- Where
- paragraph 14
- Published
- 2025-08-27
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-what-scaling-does-and-doesnt-buy-c1e720-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-what-scaling-does-and-doesnt-buy-c1e720-a2.