“GPT-4 will not have reliable models of the things that it talks about that are accessible to external programmers in a way that reliably feeds downstream processes.”
Our reading, the sentence that was checked:
GPT-4 will not have reliable models of the things it talks about that are accessible to external programmers in a way that reliably feeds downstream processes.
Assessment: Unresolved No deadline stated on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, What to Expect When You’re Expecting … GPT-4, prediction 4 of 7.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-what-to-expect-when-youre-expecting-37868f-a1
We recorded 27 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-12-25. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No deadline stated
- Deadline state on 2026-10-29
- No deadline stated (the release’s reference date)
Conditions
- accessible to external programmers in a way that reliably feeds downstream processes (qualifier, stated by the speaker)
From the speaker
- GPT-4
- reliable models of the things that it talks about
- accessible to external programmers
- reliably feeds downstream processes
Our interpretation
- Models of things means usable representations of entities, relations, states, or actions that programmers can obtain and use.
- The claim does not explicitly require a symbolic internal architecture or access to model weights.
- Reliability concerns the correctness and downstream usefulness of representations, rather than their formatting alone.
- Corrected representations are relevant counterevidence, but their reliability cannot automatically be attributed to GPT-4 when external validators or people supply essential corrections.
- Accessibility is part of the claimed capability, not a conditional antecedent.
Counts as supported if: Interface documentation and representative evaluations establish that programmers cannot obtain representations sufficiently correct for dependable downstream use without substantial semantic repair.
Counts as contradicted if: Programmers demonstrably obtain semantically reliable representations from GPT-4 that feed downstream processes with measured practical success, with the source of any corrections clearly accounted for.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No deadline stated
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
GPT-4 has been used to produce explicit world models for downstream planning, so the evidence does not support a categorical claim that programmers cannot obtain such representations. Those models also required substantial factual correction. Whether this settles the forecast depends on an unstated rule about external repair and on the breadth required of a reliable model. The cited implementation therefore does not meet a decisive resolution rule.
Limitations
- The meaning of accessible models is ambiguous.
- Success with corrected planning models does not establish that GPT-4's initial representations were reliable.
- The study tests a limited set of domains.
Evidence (2 items)
Context only: OpenAI,
it generates text outputs
Why it matters here: The launch interface exposes generated text, which can include code. That description does not establish either the presence or the absence of reliable representations obtainable through prompting.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Context only: NeurIPS 2023; author manuscript on arXiv,
59 feedback messages
Why it matters here: Researchers extracted explicit action models from GPT-4 and used corrected models to solve downstream planning tasks. The substantial corrective feedback supports the reliability concern while the successful extraction and downstream use challenge a categorical absence of accessible representations.
- Publisher
- NeurIPS 2023; author manuscript on arXiv
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
What would settle it
A reproducible study measuring semantic correctness and downstream reliability of GPT-4-derived representations across representative domains, with external corrections separately measured and the permitted correction level specified.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, What to Expect When You’re Expecting … GPT-4
GPT-4 will not have reliable models of the things that it talks about that are accessible to external programmers in a way that reliably feeds downstream processes.
- Where
- prediction 4 of 7
- Published
- 2022-12-25
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-what-to-expect-when-youre-expecting-37868f-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-what-to-expect-when-youre-expecting-37868f-a1.