“Back on planet earth, according to December’s Remote Labor Index, only 2.5% of human “online tasks” (physical tasks were excluded) could be done by AI.”
Our reading, the sentence that was checked:
According to the December 2025 Remote Labor Index, AI could do only 2.5% of human online tasks.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Promises are cheap, paragraph 6.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-promises-are-cheap-a44926-a1
We recorded 8 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2026-02-12. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- physical tasks excluded (qualifier, stated by the speaker)
From the speaker
- according to December’s Remote Labor Index
- only 2.5%
- human “online tasks”
- physical tasks were excluded
- could be done by AI
Our interpretation
- December means December 2025.
- Because Marcus attributes the figure to a named benchmark, the ordinary scoped reading concerns its evaluated remote freelance projects and tested agents.
- Could be done means completed end to end at the benchmark’s required professional quality.
- December’s is read as the results reported during December, not necessarily a new study or new test runs conducted that month.
- The figure is not treated as a universal ceiling on all online tasks or all possible AI configurations.
Counts as supported if: The RLI results reported during December 2025 identify approximately 2.5 percent as the highest automation rate among evaluated agents on its remote-work projects.
Counts as contradicted if: The comparable December results report a materially different highest automation rate, or 2.5 percent measures something other than acceptable completion of the evaluated remote-work projects.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
December reporting identifies 2.5 percent as the best RLI completion rate, matching the original primary study. This supports Marcus’s attribution when read as describing the benchmark’s evaluated freelance projects. It does not establish a universal limit on online work. The original paper appeared in October, so the evidence supports results reported in December rather than a distinct December study.
Limitations
- The judgment depends on keeping the claim within the named benchmark’s scope.
- December timing is established through a university’s dated summary; the primary result is from October.
- Whole-project completion is different from partial assistance or success on smaller component tasks.
- A stronger reading covering all online work would remain unproved.
Evidence (2 items)
Supports the claim: United Nations University Campus Computing Centre,
The highest-performing AI agent, Manus, managed to complete only 2.5% of projects at a professional level.
Why it matters here: This dated December account reports the claimed top rate for RLI projects. The event date identifies December reporting of standing results, not new experiments; an institutional summary is used because a precisely dated December primary snapshot was not verified.
- Publisher
- United Nations University Campus Computing Centre
- Kind of source
- Secondary summary
- Describes events of
- Public from
Context only: Mazeika and colleagues, Center for AI Safety and Scale AI, arXiv,
AI agents perform near the floor on RLI, with the highest-performing agent achieving an automation rate of 2.5%.
Why it matters here: The primary paper confirms the original numerical result. It is context because it predates the December target period and does not independently establish a December update.
- Publisher
- Mazeika and colleagues, Center for AI Safety and Scale AI, arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Promises are cheap
Back on planet earth, according to December’s Remote Labor Index, only 2.5% of human “online tasks” (physical tasks were excluded) could be done by AI.
- Where
- paragraph 6
- Published
- 2026-02-12
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-promises-are-cheap-a44926-a1: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.