“Taking this into account, the fraction of problems where a composite AI system could solve the problem starting with its English language statement is at most 60%.”
Our reading, the sentence that was checked:
Of the 50 IMO geometry problems from 2000 to 2024, a composite AI system starting from the English problem statements could solve at most 60%.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, AlphaGeometry2: Impressive accomplishment, but still a long path ahead, paragraph 8.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-alphageometry2-impressive-accomplishment-7c1011-a1
We recorded 11 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-02-17. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- the 50 IMO geometry problems from 2000 to 2024 (scope, stated by the speaker)
- starting from the English language statement (qualifier, stated by the speaker)
- at most (qualifier, stated by the speaker)
From the speaker
- starting with its English language statement
- at most 60%
- this succeeded on 30 problems
- all the 50 IMO geometry problems from 2000 to 2024
Our interpretation
- Assess the reported English-input pipeline on the 50 benchmark entries, not a permanent ceiling on future systems.
- Translation success is a necessary intermediate result, not proof of downstream solving.
- The target period is the February 2025 experiment report; 2000–2024 identifies the source problems, not the evaluation date.
- Original competition problems and split benchmark entries must be counted consistently.
Counts as supported if: Matching per-entry results establish that no more than 30 of the 50 benchmark entries could pass the reported English-input pipeline, including correct formalization and solving.
Counts as contradicted if: Matching results establish more than 30 correctly solved benchmark entries from English statements using the described pipeline.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The reported translation count and benchmark denominator use different units. Without their mapping and downstream results, the 60% ceiling does not follow. That undermines the calculation but does not establish that the pipeline actually exceeded 60%, so contradiction would also be premature.
Limitations
- Automatic translation success does not itself establish solving success.
- An observed translation failure does not establish a permanent inability to formalize that problem.
Evidence (1 item)
Context only: arXiv / Google DeepMind authors,
30 out of 39
Why it matters here: Section 3 reports translation successes, whereas Section 8 constructs 50 benchmark entries from 45 original problems by splitting some. The report does not identify the mapping needed to turn these translation successes into the claimed end-to-end ceiling.
- Publisher
- arXiv / Google DeepMind authors
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Item-level translation and solving records mapped consistently onto all 50 benchmark entries for the reported pipeline.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, AlphaGeometry2: Impressive accomplishment, but still a long path ahead
Taking this into account, the fraction of problems where a composite AI system could solve the problem starting with its English language statement is at most 60%.
- Where
- paragraph 8
- Published
- 2025-02-17
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-alphageometry2-impressive-accomplishment-7c1011-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-alphageometry2-impressive-accomplishment-7c1011-a1.