“Here is a further example of my last point that ChatGPT’s exciting new image system has trouble with functional understanding, sent to me by a reader:”
Our reading, the sentence that was checked:
ChatGPT's exciting new image system has trouble with functional understanding.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, ChatGPT doesn’t know its whisk from its elbow, paragraph 1.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-chatgpt-doesnt-know-its-whisk-from-251653-a1
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2026-04-22. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- ChatGPT’s exciting new image system
- has trouble with functional understanding
- a further example
- sent to me by a reader
Our interpretation
- The timing identifies the target as ChatGPT Images 2.0, introduced on April 21, 2026.
- The plain reading is that the system makes recurring mistakes depicting how objects work or are used. It does not assert that every output fails or that the system has no useful capabilities.
- Because this describes an ongoing capability without a reporting period, assess the newly released system from April 21 through the April 22 post. This is the latest publicly observable period before publication.
- Assess observable output errors, without treating them as proof of a particular internal cognitive mechanism.
Counts as supported if: Documented tests or independently verifiable outputs attributable to the system in the target period show recurring errors in ordinary object operation or use, with prompts and outputs sufficient to distinguish functional mistakes from deliberately unusual requests.
Counts as contradicted if: Broad, reproducible tests of ordinary object operation and use show reliable functional depictions, and attempts to reproduce the alleged difficulty establish that it arose from misleading prompts, misidentified models, or isolated artifacts rather than a recurring limitation.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The claim has an ordinary, testable reading: recurring mistakes depicting how objects work or are used. The search did not locate independent evidence establishing those mistakes in the launch-period system. Later evidence includes a correct violin bow grip and strong physical-editing benchmark results, but neither settles performance on April 21–22. Marcus's assertion and the unavailable reader-submitted example cannot supply independent confirmation. The evidence therefore supports neither a positive nor a negative verdict.
Limitations
- The assessment concerns observable functional depictions, not whether the model possesses human-like understanding.
- No generation tests were conducted; current tests would not by themselves establish historical performance.
- The target period is inferred from the ongoing observation and release timing.
Evidence (3 items)
Context only: OpenAI,
We’re introducing ChatGPT Images 2.0, our new image generation model in ChatGPT.
Why it matters here: Identifies the newly introduced image system at the time of Marcus's post. It does not measure functional errors.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: Tom's Guide,
The bow hand grip looks technically correct.
Why it matters here: The reviewer reports a correct violin bow grip, a relevant functional success. The article does not establish an April 21–22 test date, and one success cannot exclude recurring difficulties elsewhere.
- Publisher
- Tom's Guide
- Kind of source
- Demonstration
- Describes events of
- Public from
Context only: PhyEditBench authors,
It won first place on the list of normal data!
Why it matters here: The benchmark reports strong performance on normal physical image editing, qualifying any sweeping inference of functional incapacity. These later results cover a different test period and cannot settle the launch-period observation.
- Publisher
- PhyEditBench authors
- Kind of source
- Benchmark result
- Describes events of
- Public from
What would settle it
Dated launch-period prompts and outputs, with model and mode identified, showing repeated functional-use failures or reliable performance across a suitable range of ordinary tasks. Later analysis could qualify if it verifies those historical records.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, ChatGPT doesn’t know its whisk from its elbow
Here is a further example of my last point that ChatGPT’s exciting new image system has trouble with functional understanding, sent to me by a reader:
- Where
- paragraph 1
- Published
- 2026-04-22
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-chatgpt-doesnt-know-its-whisk-from-251653-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-chatgpt-doesnt-know-its-whisk-from-251653-a1.