“GPT”s heavy use of a technique called embeddings makes it really good at substituting synonyms and more broadly related phrases, but the same tendency towards substitution often lead it astray.”
Our reading, the sentence that was checked:
GPT's reliance on embeddings makes it good at substituting related phrases, and the same tendency often causes it to make mistakes.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, How come GPT can seem so brilliant one minute and so breathtakingly dumb the next?, paragraph 38.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-how-come-gpt-can-seem-so-brilliant-57661c-a1
We recorded 9 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-12-01. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Causal claim
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- heavy use of a technique called embeddings
- really good at substituting synonyms and more broadly related phrases
- the same tendency towards substitution
- often lead it astray
Our interpretation
- GPT means the GPT-3-era models discussed in the post.
- The claim attributes both useful substitutions and recurrent errors to a shared embedding-driven substitution mechanism.
- Often means a recurring, material source of errors; the post supplies no numerical frequency.
- Semantic similarity between embeddings, by itself, does not establish that substitutions cause factual errors.
Counts as supported if: Controlled interventions establish that embedding-based semantic substitution improves appropriate paraphrasing while also causing a recurring class of incorrect substitutions in the relevant GPT models.
Counts as contradicted if: Controlled studies directly testing the proposed pathway find that it does not cause the alleged errors, or that the supposed substitutions are not generated by that mechanism.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Does the evidence support the cause?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The located mechanistic research studies how models store and retrieve factual associations. It does not establish Marcus's proposed causal chain from embeddings to useful substitutions and then to recurring mistakes. The existence of semantic similarity or factual errors would not settle that causal claim.
Limitations
- Finding another mechanism involved in factual predictions does not disprove a contribution from embeddings.
- The claim's frequency term has no stated denominator.
Evidence (1 item)
Context only: arXiv; Kevin Meng and colleagues,
mediate factual predictions
Why it matters here: The study identifies causal mechanisms for factual predictions and edits factual associations. Its findings concern GPT-2 and related open models and do not test whether synonym substitution causes recurring GPT-3 errors.
- Publisher
- arXiv; Kevin Meng and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
An intervention study that isolates semantic substitution in a relevant GPT model and measures its causal contribution to both correct paraphrasing and errors.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, How come GPT can seem so brilliant one minute and so breathtakingly dumb the next?
GPT”s heavy use of a technique called embeddings makes it really good at substituting synonyms and more broadly related phrases, but the same tendency towards substitution often lead it astray.
- Where
- paragraph 38
- Published
- 2022-12-01
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-how-come-gpt-can-seem-so-brilliant-57661c-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-how-come-gpt-can-seem-so-brilliant-57661c-a1.