“On LLMs China is already catching up.”
Our reading, the sentence that was checked:
China is already catching up with the U.S. on LLMs.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Hot take on an AI catfight, paragraph 10.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-hot-take-on-an-ai-catfight-2227f1-a1
We recorded 6 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-01-22. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- On LLMs
- China
- already catching up
Our interpretation
- Read as narrowing the capability gap between leading Chinese and U.S. language models, not claiming that China already led.
- Use comparable language, coding and mathematical-reasoning benchmarks as ordinary measures of LLM capability.
- Assess the latest models available before the post and the preceding year's trend, using earlier results as the baseline.
- Later publications may establish historical results, but post-January 22, 2025 model improvements do not count.
Counts as supported if: Comparable evaluations show that leading Chinese models materially narrowed their performance gap with leading U.S. models across several relevant benchmarks by the post date.
Counts as contradicted if: Comparable evaluations show no meaningful narrowing, or a widening gap, across the relevant capability measures during that period.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Stanford's retrospective comparison shows substantial narrowing across language, mathematics and coding benchmarks during 2024, before Marcus's post. DeepSeek's December 2024 report provides contemporaneous corroboration. These results support 'catching up'; they do not establish Chinese leadership across every capability or across the wider AI industry.
Limitations
- Benchmark performance does not capture every aspect of real-world reliability.
- The comparison concerns leading models.
- DeepSeek's corroborating evaluation is developer-reported.
- The main comparative report was published after the post but measures pre-post outcomes.
Evidence (2 items)
Supports the claim: Stanford Institute for Human-Centered Artificial Intelligence,
narrowed substantially
Why it matters here: The preceding sentence gives end-2023 U.S.–China gaps of 17.5, 13.5, 24.3 and 31.6 percentage points on MMLU, MMMU, MATH and HumanEval. All four narrowed by end-2024, including the language, mathematics and coding measures directly relevant to this claim.
- Publisher
- Stanford Institute for Human-Centered Artificial Intelligence
- Kind of source
- Benchmark result
- Describes events of
- Public from
Supports the claim: DeepSeek-AI, via arXiv,
performance comparable
Why it matters here: The contemporaneous report corroborates that a leading Chinese model had approached leading closed-model performance before the post. It is developer-reported evidence and is secondary in weight to the cross-country benchmark comparison.
- Where
- Abstract
- Publisher
- DeepSeek-AI, via arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Hot take on an AI catfight
On LLMs China is already catching up.
- Where
- paragraph 10
- Published
- 2025-01-22
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-hot-take-on-an-ai-catfight-2227f1-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-hot-take-on-an-ai-catfight-2227f1-a1.