“AI “Agents” will be endlessly hyped throughout 2025 but far from reliable, except possibly in very narrow use cases.”
Our reading, the sentence that was checked:
Throughout 2025, AI agents will be endlessly hyped but far from reliable, except possibly in very narrow use cases.
Assessment: Supported Deadline passed on 2026-10-29 Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, 25 AI Predictions for 2025, from Marcus on AI, under the heading 'High confidence', in '25 Predictions for 2025'.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment gm-agents-hyped-unreliable-2025-a3
We recorded 37 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-01-01. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Forecast
- Deadline, as the speaker wrote it
- “throughout 2025 (post dated 2025-01-01, under 'High confidence')”
- Deadline state when assessed
- Deadline passed
- Deadline state on 2026-10-29
- Deadline passed (the release’s reference date)
From the speaker
- AI “Agents” will be endlessly hyped throughout 2025
- but far from reliable
- except possibly in very narrow use cases
- listed under the heading 'High confidence'
- 'my own predictions for where we will be at the end of this year', post dated 2025-01-01
Our interpretation
- We read 'endlessly hyped' as leading AI companies promoting agents at points spread across 2025, including its last quarter.
- We read 'far from reliable' as independent tests from late 2025 showing general-purpose agents failing a large share of realistic multi-step work tasks. Narrow uses are excepted by the speaker.
- Evidence from earlier in 2025 is shown too, but the end of the year must be covered.
Counts as supported if: Leading AI companies promoted agents at points spread across 2025, including its last quarter, and independent tests published late in 2025 show general-purpose agents failing a large share of realistic multi-step work tasks.
Counts as contradicted if: Independent tests published late in 2025 show general-purpose agents completing most realistic multi-step work tasks across broad uses, or agents were not prominently promoted during 2025.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Did the forecast come true?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
- Evidence as of
- Deadline state when assessed
- Deadline passed
- Assessment recorded
- Policy version
- 2026-10-28.5 (the current policy is 2026-10-28.7; this assessment has not been rechecked under it)
Rationale
On January 1, 2025, under 'High confidence', Marcus predicted that AI agents would be endlessly hyped throughout 2025 but far from reliable, except possibly in very narrow use cases. OpenAI promoted agents in January, June, and October 2025, from saying they may 'join the workforce' to launching tools to build them. Independent benchmarks found the best agent completed 30% of simulated workplace tasks (September 2025) and 2.5% of real remote-work projects (30 October 2025).
Limitations
- The promotion shown comes from one company, OpenAI. It does not measure how much agents were promoted across the industry.
- Both benchmarks are preprints, and the latest is from 30 October 2025, not the end of the year.
- Whether an agent use counts as a 'very narrow use case' is not defined by the speaker.
Evidence (5 items)
Supports the claim: Sam Altman (OpenAI CEO), personal blog,
in 2025, we may see the first AI agents “join the workforce” and materially change the output of companies.
Why it matters here: Early in 2025, OpenAI's CEO said AI agents may 'join the workforce' that year. This shows agents promoted as transformative at the start of 2025. It is a company statement about its own field, not a test of agents.
- Publisher
- Sam Altman (OpenAI CEO), personal blog
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Supports the claim: Sam Altman (OpenAI CEO), personal blog,
2025 has seen the arrival of agents that can do real cognitive work; writing computer code will never be the same.
Why it matters here: In June 2025, OpenAI's CEO said 2025 had seen agents that can do real cognitive work. This shows the promotion continuing mid-year. It is a company claim, not an independent test; its example, writing code, is one use.
- Publisher
- Sam Altman (OpenAI CEO), personal blog
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Supports the claim: arXiv (Carnegie Mellon University and others),
the most competitive agent can complete 30% of tasks autonomously.
Why it matters here: An independent benchmark of agents on tasks from a simulated software company, revised in September 2025: the best agent completed 30% of tasks on its own, and harder long-horizon tasks were still beyond reach. Simpler tasks were often solved. This is a preprint.
- Publisher
- arXiv (Carnegie Mellon University and others)
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: OpenAI,
Today we’re launching AgentKit, a complete set of tools for developers and enterprises to build, deploy, and optimize agents.
Why it matters here: In October 2025, OpenAI launched a product line for building and deploying agents. This shows agents still being promoted in the last quarter of 2025. It is a company announcement, not a test of agents.
- Publisher
- OpenAI
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
Supports the claim: arXiv (Remote Labor Index authors),
AI agents perform near the floor on RLI, with the highest-performing agent achieving an automation rate of 2.5%.
Why it matters here: An independent benchmark of real, paid remote-work projects across many sectors, published 30 October 2025. The best agent completed 2.5% of projects to the required standard. This is a preprint.
- Publisher
- arXiv (Remote Labor Index authors)
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, 25 AI Predictions for 2025, from Marcus on AI
AI “Agents” will be endlessly hyped throughout 2025 but far from reliable, except possibly in very narrow use cases.
- Published
- 2025-01-01
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment gm-agents-hyped-unreliable-2025-a3: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Site correctionBy this site
A new assessment with the same outcome replaced the previous one.
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment gm-agents-hyped-unreliable-2025-a2, which was not published.