“They still aren’t adding a lot of quantifiable value to the world:”
Our reading, the sentence that was checked:
LLMs still are not adding much quantifiable value to the world.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Let’s be honest, Generative AI isn’t going all that well, paragraph 4.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-lets-be-honest-generative-ai-isnt-17889c-a1
We recorded 7 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2026-01-12. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- They still
- aren’t adding a lot
- quantifiable value
- to the world
Our interpretation
- They refers to LLMs.
- The plain reading is that realized, measurable benefits remain modest in aggregate, including productivity, cost savings, and other useful outcomes.
- This permits benefits in individual applications and does not require complete job automation.
- Use the latest completed quarter before publication, October–December 2025, for the ongoing state. Earlier measurements provide context.
- No monetary threshold or worldwide accounting measure is supplied; none is invented.
Counts as supported if: Broad measurements of realized LLM benefits show consistently modest aggregate gains across major uses, with adequate coverage and attribution.
Counts as contradicted if: Broad measurements establish substantial realized aggregate benefits attributable to LLM use during the target period.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The record contains measurable reported benefits and a controlled example of slower work. Neither establishes the size of worldwide LLM benefits in late 2025. The survey evidence challenges a broad dismissal of value, but it is self-reported, US-specific, and covers generative AI more broadly. The negative experiment is also narrow. Low autonomous job completion cannot substitute for measuring value from assistance.
Limitations
- The claim lacks a threshold and an agreed measure of value.
- The cited measurements concern earlier periods and narrower populations.
- The evidence does not establish a worldwide total after costs.
Evidence (2 items)
Context only: Federal Reserve Bank of St. Louis,
1.6% of all work hours.
Why it matters here: Pooled February, May, and August surveys provide material evidence against dismissing measurable benefits. These are self-reported US savings for generative AI, not a causal worldwide measure of realized LLM value in the target quarter.
- Publisher
- Federal Reserve Bank of St. Louis
- Kind of source
- Independent measurement
- Describes events of
- Public from
Context only: METR / arXiv,
AI tooling slowed developers down.
Why it matters here: The February–June experiment demonstrates that benefits were not universal and that perceived savings could differ from measured outcomes. Its experienced developers, particular projects, and earlier models cannot establish worldwide value in late 2025.
- Where
- Abstract
- Publisher
- METR / arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Representative measurements of realized late-2025 LLM benefits across major uses, with credible attribution and cost treatment, and a defensible interpretation of substantial aggregate value.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Let’s be honest, Generative AI isn’t going all that well
They still aren’t adding a lot of quantifiable value to the world:
- Where
- paragraph 4
- Published
- 2026-01-12
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-lets-be-honest-generative-ai-isnt-17889c-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-lets-be-honest-generative-ai-isnt-17889c-a1.