“GPT-4 can’t even play a good game of chess (it’s about as good as chess computers from 1978).”
Our reading, the sentence that was checked:
GPT-4 cannot play a good game of chess and is about as good as 1978 chess computers.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, GPT-5 and irrational exuberance, paragraph 6.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-gpt-5-and-irrational-exuberance-d99615-a2
We recorded 17 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2023-04-08. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- GPT-4
- can’t even play a good game of chess
- about as good as chess computers from 1978
Our interpretation
- The target is the original publicly available GPT-4 during March 2023 through the post date, using the latest completed month and current partial month.
- Playing a good game means sustained competent, legal play across complete games, rather than answering isolated chess puzzles.
- The ordinary comparison concerns GPT-4 choosing moves without a separate chess engine.
- About as good requires broadly comparable playing strength under comparable conditions. Marcus supplies neither a particular 1978 machine nor a tolerance.
Counts as supported if: Reproducible tests of the contemporary GPT-4 show weak full-game play and establish approximate parity with identified, representative 1978 chess computers under comparable conditions.
Counts as contradicted if: Reliable contemporary tests establish sustained competent full-game play materially stronger than representative 1978 computers, or otherwise clearly refute a specified component.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Harth's March 16, 2023 game record shows GPT-4 blunders followed by illegal move suggestions [e1]. Brereton's March 17 account reports numerous illegal moves in two attempted GPT-4 games and explicitly says the data are too limited for meaningful conclusions [e2]. These examples establish failures in those interactions, but do not settle GPT-4's general playing strength [e1, e2]. Neither account supplies a comparable evaluation against identified 1978 chess computers, leaving the historical comparison unresolved [e1, e2].
Limitations
- The admitted GPT-4 evidence consists of one recorded game and an account of two attempted games [e1, e2].
- These accounts do not establish performance across prompting methods or provide a matched historical baseline [e1, e2].
Evidence (2 items)
Supports the claim: Rafael Harth / LessWrong,
just giving stuff away
Why it matters here: The original game record documents weak play followed by illegal moves. It supports the weakness component in this interaction, but one game cannot establish general strength or parity with 1978 computers.
- Publisher
- Rafael Harth / LessWrong
- Kind of source
- Demonstration
- Describes events of
- Public from
Context only: Dmitri Brereton / DKB Blog,
the data is too limited
Why it matters here: The author explicitly limits his GPT-4 inference after only two attempted games with numerous illegal moves. The headline rating of 1402 comes from a different experiment using GPT-3.5 and cannot settle this claim.
- Where
- GPT4 Sucks
- Publisher
- Dmitri Brereton / DKB Blog
- Kind of source
- Independent measurement
- Describes events of
- Public from
What would settle it
A sufficiently large, reproducible evaluation of the original March–April 2023 GPT-4, with full game records and declared prompts, board-state assistance, illegal-move rules and time controls, compared with identified representative 1978 systems under matched conditions and explicit strength thresholds.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, GPT-5 and irrational exuberance
GPT-4 can’t even play a good game of chess (it’s about as good as chess computers from 1978).
- Where
- paragraph 6
- Published
- 2023-04-08
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-gpt-5-and-irrational-exuberance-d99615-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-gpt-5-and-irrational-exuberance-d99615-a2.