“Reports that language models have produced correct proofs for isolated open problems in mathematics, including specific Erdős problems, do not alter this assessment.”
Our reading, the sentence that was checked:
LLM proofs of isolated open math problems, including Erdős problems, do not show general intelligence.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Rumors of AGI’s arrival have been greatly exaggerated, paragraph 30.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-rumors-of-agis-arrival-have-been-197ebe-a1
We recorded 18 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2026-02-17. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- reports that language models have produced correct proofs
- isolated open problems in mathematics
- including specific Erdős problems
- do not alter this assessment
Our interpretation
- Plain reading: the LLM proofs of open problems reported up to the post (February 2026) were few and mostly on easier or obscure problems. They did not show broad, domain-general problem-solving that would overturn the judgment that AI falls short of AGI.
- The checkable core is how these results were characterized by expert mathematicians tracking them at the time, especially Terence Tao, whom the post cites.
- Target period: from late 2025 up to the post date (2026-02-17), when these reports appeared. Later progress is context only.
Counts as supported if: Expert tallies from the period show AI successes on Erdős problems were a small share, concentrated at the easy or obscure end, often relying on existing literature, and needing human checking.
Counts as contradicted if: Expert tallies from the period show AI solving a broad range of hard, central open problems autonomously, including problems typical of the set's difficulty.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The checkable part is whether the AI proofs reported up to February 2026 were isolated results that do not show broad problem-solving. Terence Tao, who tracked them closely, counted eight problems with meaningful autonomous AI progress as of January 2026, plus six more built on existing literature. He described the successes as coming from the 'long tail' of obscure problems, many with straightforward solutions. He put the true success rate at a point or two, concentrated at the easy end, with no sign that typical problems were within reach. That matches the post's description. AI did make bigger gains later in 2026, including on flagship bounds by September, but that is after the period the claim describes. Whether the results bear on AGI is partly a judgment, but the facts it rests on hold for February 2026.
Limitations
- The 'do not alter this assessment' part is an evaluative judgment.
- The claim is time-bound: later 2026 results were stronger.
- The Tao summary page is a living document compiled with AI help, though reviewed by Tao.
Evidence (4 items)
Supports the claim: TechCrunch,
Tao conjectured that the scalable nature of AI systems makes them “better suited for being systematically applied to the ‘long tail’ of obscure Erdős problems, many of which actually have straightforward solutions.”
Why it matters here: Tao described the AI successes as falling on obscure problems that often have straightforward solutions. This matches the post's point that these results do not show deep, general problem-solving.
- Where
- paragraph 8
- Publisher
- TechCrunch
- Kind of source
- News report
- Describes events of
- Public from
Supports the claim: TechCrunch,
counting eight different problems where AI models made meaningful autonomous progress on an Erdős problem, with six other cases where progress was made by locating and building on previous research. It’s a long way from AI systems being able to do math without human intervention
Why it matters here: Only a handful of autonomous successes were recorded at the time, some built on existing literature, and humans were still needed. This supports 'isolated'.
- Where
- paragraph 7
- Publisher
- TechCrunch
- Kind of source
- News report
- Describes events of
- Public from
Supports the claim: Terence Tao (tao-web, AI-compiled summary reviewed by Tao),
on the Erdős problems, as he measured it in January 2026, the true success rate was only a point or two — non-trivial in absolute count across a thousand-plus problems, but concentrated at the easy end, with no evidence at that point that the median problem was in reach.
Why it matters here: Tao's January 2026 estimate was a 1-2% success rate, concentrated on easy problems. This backs the claim that the successes were isolated and not evidence of general competence.
- Publisher
- Terence Tao (tao-web, AI-compiled summary reviewed by Tao)
- Kind of source
- Secondary summary
- Describes events of
- Public from
Context only: Terence Tao (tao-web, AI-compiled summary reviewed by Tao),
as of early 2026, yet to make real progress. That last qualification eroded quickly: by September 2026 he was reporting AI systems improving flagship bounds (bounded gaps between primes)
Why it matters here: Confirms there was no progress on marquee problems in early 2026. It also notes stronger AI results by September 2026, which fall outside the claim's period.
- Publisher
- Terence Tao (tao-web, AI-compiled summary reviewed by Tao)
- Kind of source
- Secondary summary
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Rumors of AGI’s arrival have been greatly exaggerated
Reports that language models have produced correct proofs for isolated open problems in mathematics, including specific Erdős problems, do not alter this assessment.
- Where
- paragraph 30
- Published
- 2026-02-17
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-rumors-of-agis-arrival-have-been-197ebe-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-rumors-of-agis-arrival-have-been-197ebe-a1.