“The graph pertains only to software-development tasks. Not general intelligence.”
Our reading, the sentence that was checked:
The METR time-horizon graph pertains only to software-development tasks, not general intelligence.
Assessment: Mixed Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Misplaced panic over AI progress, paragraph 20.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-misplaced-panic-over-ai-progress-aa49a6-a2
We recorded 14 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2026-05-10. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- The graph
- only
- software-development tasks
- Not general intelligence
Our interpretation
- Assess the task coverage and interpretation of the graph available when Marcus posted.
- Use the May 8–10 graph snapshot, the latest published update before the post.
- Read software development ordinarily as developing and maintaining software. Do not silently expand only to encompass every technical task performed through a computer.
- Read not general intelligence as a limitation on what this benchmark directly measures, not as a claim that its results contain no information about broader capabilities.
Counts as supported if: The task inventory is confined to software development, and the graph does not directly establish general intelligence.
Counts as contradicted if: The inventory includes materially distinct task domains, contradicting only software development; evidence of broad, representative capability measurement would separately contradict the second clause.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Mixed
Part of the claim holds and part does not, and the record says which part is which.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The warning about general intelligence holds: METR describes a restricted benchmark whose tasks differ from ordinary work. The word only is too strong under a literal reading of software development. METR explicitly includes machine learning and cybersecurity. If Marcus intended software development as shorthand for that whole technical cluster, the disagreement is largely terminological; that interpretation is flagged.
Limitations
- The boundary between software development, ML engineering, and cybersecurity is not sharp.
- The documentation read is the September 8 version; its update history identifies May 8 as the relevant graph update.
- A restricted benchmark may still provide indirect evidence about broader capabilities.
Evidence (2 items)
Supports the claim: METR,
Most jobs
Why it matters here: METR explains that its task distribution omits important features of ordinary work. Together with the documented domain restrictions, this supports the narrower point that the graph is not a direct demonstration of general intelligence.
- Publisher
- METR
- Kind of source
- Benchmark result
- Describes events of
- Public from
Goes against the claim: METR,
cybersecurity
Why it matters here: METR describes the TH1.1 graph's distribution as spanning three technical domains. That contradicts the exclusive wording under the ordinary, narrower meaning of software development.
- Publisher
- METR
- Kind of source
- Benchmark result
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Misplaced panic over AI progress
The graph pertains only to software-development tasks. Not general intelligence.
- Where
- paragraph 20
- Published
- 2026-05-10
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-misplaced-panic-over-ai-progress-aa49a6-a2: Current. Outcome: Mixed. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-misplaced-panic-over-ai-progress-aa49a6-a2.
HeldBy this site
The anthropic review could not run (error).
About assessment marcus-misplaced-panic-over-ai-progress-aa49a6-a1, which was not published.