“Essentially none of the fancy cognitive things GPT-4 is rumored to do stand up.”
Our reading, the sentence that was checked:
Essentially none of the advanced cognitive abilities attributed to GPT-4 hold up under scrutiny.
Assessment: Contradicted Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, GPT-5 and irrational exuberance, paragraph 42.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-gpt-5-and-irrational-exuberance-28c621-a1
We recorded 17 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2023-04-08. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- Essentially none
- fancy cognitive things
- GPT-4
- stand up
Our interpretation
- The ordinary reading is that almost all attributed advanced cognitive capabilities fail careful testing.
- Controlled success on reasoning tasks counts as a capability surviving scrutiny; it does not establish human-like internal thought or general intelligence.
- I treat cognitive reflection and reasoning about conjunctive probabilities as examples of the cognitive reasoning abilities covered by the broad wording.
- I use March 2023 through the post date as the latest reporting window for the initial GPT-4 release. Later publications may establish results collected in that window.
Counts as supported if: Controlled investigations of the principal attributed cognitive abilities find that apparent competence largely disappears under adequate testing.
Counts as contradicted if: Controlled investigations of contemporary GPT-4 establish substantial competence across multiple cognitive reasoning tasks beyond isolated demonstrations, making the near-total dismissal untenable.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Contradicted
The dated evidence goes against what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Under the ordinary behavioral reading, the near-total dismissal is too broad. Controlled March 2023 experiments found substantial GPT-4 competence on two reasoning paradigms using new task variants and repeated trials. Documented planning failures remain important, but do not erase those successes. This does not establish general intelligence or mastery of theory of mind.
Limitations
- The result depends on counting controlled reasoning performance among the unspecified 'fancy cognitive things'.
- The decisive study was published after the post, although its relevant experiments occurred in March 2023.
- The tests cover limited forms of reasoning and do not settle every attributed cognitive ability.
Evidence (2 items)
Goes against the claim: Communications Psychology / Europe PMC,
reasoning tests.
Why it matters here: The study tested new cognitive-reflection and Linda/Bill problems, finding strong GPT-4 performance across repeated trials. Figure 5 explicitly dates these experiments to March 2023, so the later publication bears on the contemporary model.
- Publisher
- Communications Psychology / Europe PMC
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
Context only: Microsoft Research / arXiv,
the model often fails.
Why it matters here: The researchers document arithmetic and planning failures, qualifying sweeping claims of intelligence. They studied an early development version, so these results are contextual rather than decisive measurements of the released model.
- Publisher
- Microsoft Research / arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, GPT-5 and irrational exuberance
Essentially none of the fancy cognitive things GPT-4 is rumored to do stand up.
- Where
- paragraph 42
- Published
- 2023-04-08
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-gpt-5-and-irrational-exuberance-28c621-a1: Current. Outcome: Contradicted. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-gpt-5-and-irrational-exuberance-28c621-a1.