โSuch pretraining is not uncommon in the field, but was not made clear in the video.โ
Our reading, the sentence that was checked:
Such pretraining is not uncommon in the field, but OpenAI's video did not make clear that o3 had been pretrained on ARC examples.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-c39-2d2868-a1
We recorded 17 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-12-22. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the releaseโs reference date)
From the speaker
- Such pretraining is not uncommon in the field
- was not made clear in the video
Our interpretation
- The video is OpenAI's December 20, 2024 o3 announcement.
- The plain reading requires an understandable disclosure that the evaluated o3 had trained on public ARC examples; a technical label alone may not establish clarity.
- Not uncommon means an established practice with multiple examples, not necessarily a majority practice.
- Training before evaluation must be distinguished from adaptation using a test task's own demonstrations.
Counts as supported if: Contemporaneous methods establish that preparatory ARC training was an established practice, and the complete recording shows no clear spoken or readable explanation of o3's exposure to public ARC training tasks.
Counts as contradicted if: The recording clearly explains that exposure, or reliable contemporaneous evidence contradicts the assertion that such preparation was an established practice.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The technical report provides relevant examples of preparatory training. The central disclosure claim remains unsettled. The original video did not yield usable text, and accessible summaries cannot prove an omission. The secondary transcript is unreliable at the critical passage. These retrieval failures are not evidence that OpenAI failed to disclose the training.
Limitations
- Clarity requires considering slides as well as spoken words.
- Established use of test-time training alone would not establish Marcus's claim about preparation before evaluation.
Evidence (1 item)
Context only: Franรงois Chollet and colleagues, arXiv,
OmniARC (Barbadillo [3]): A Qwen2.5-0.5B-Instruct model pretrained on multiple program induction tasks and further fine-tuned at test time with augmented ARC data.
Why it matters here: The report describes preparatory training and test-time adaptation as separate components of contemporary ARC systems. It provides background for the practice claim but says nothing about disclosure in the later video.
- Publisher
- Franรงois Chollet and colleagues, arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
A verified recording transcript with readable slide captures, together with contemporaneous descriptions of comparable preparatory training methods.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, ๐ผ๐ฏ โ๐๐ฅ๐ ๐๐๐โ ๐ฝ๐ผ๐๐๐บ๐ผ๐ฟ๐๐ฒ๐บ ๐บ๐ฒ๐ด๐ฎ๐๐ต๐ฟ๐ฒ๐ฎ๐ฑ: ๐๐ต๐ ๐๐ต๐ถ๐ป๐ด๐ ๐ด๐ผ๐ ๐ต๐ฒ๐ฎ๐๐ฒ๐ฑ, ๐๐ต๐ฎ๐ ๐๐ฒ๐ป๐ ๐๐ฟ๐ผ๐ป๐ด, ๐ฎ๐ป๐ฑ ๐๐ต๐ฎ๐ ๐ถ๐ ๐ฎ๐น๐น ๐บ๐ฒ๐ฎ๐ป๐
Such pretraining is not uncommon in the field, but was not made clear in the video.
- Where
- paragraph 6
- Published
- 2024-12-22
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-c39-2d2868-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.