AI Agents have, so far, mostly been a dud
We recorded 21 claims from this post. 7 were checked against evidence. 14 can’t be checked. 9 more are held. Read the post at its source.
Claims from this post (21)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
Observation
“All the big companies have in fact introduced agents.”
Our reading: All the big AI companies have introduced AI agents as of August 2025.
No horizon Evidence as of
Limitation: The company universe is undefined.
Assessment: UnresolvedSaid
Observation
“But, none, to my knowledge are reliable, except maybe in very narrow uses cases.”
Our reading: To his knowledge, none of the big companies' AI agents are reliable, except maybe in very narrow use cases.
No horizon Evidence as of
Limitation: The July reports do not provide representative, repeated trials or systematic rates of consequential errors and human intervention [e2, e3].
Assessment: UnresolvedSaid
Causal claim
“In practice, those mistakes and errors have greatly limited the utility agents like ChatGPT agent.”
Our reading: Mistakes and errors have greatly limited the utility of agents like ChatGPT agent.
No horizon Evidence as of
Limitation: The causal evidence consists of small, selected task demonstrations.
Assessment: Can’t be checkedSaid
Observation
“The “pushing the boundaries of what’s possible” that Roose promised us? Largely absent.”
Our reading: AI agents in 2025 have largely not delivered the boundary-pushing capabilities that Kevin Roose promised.
No horizon
Assessment: UnresolvedSaid
Observation
“That sentiment that has only grown over the last several months.”
Our reading: The sentiment that AI agents' reality doesn't match the hype has grown from March to August 2025.
No horizon Evidence as of
Limitation: Trust and disappointment with hype are different measures.
Assessment: Can’t be checkedSaid
Forecast
“(In fairness, ChatGPT agent wasn’t yet out when Penrose ran the test, but I would be surprised if their results were wildly different).”
Our reading: ChatGPT agent is unlikely to produce wildly different results on Penrose's account-balance benchmark.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“found a result that I suspect will be typical: AI errors tend to compound over time.”
Our reading: He suspects it will be typical that AI agent errors compound over time, as in Penrose's account-balance benchmark.
No horizon
Qualifier: I suspect
Assessment: SupportedSaid
Observation
“The unrelenting hallucinations problem frequently rears its ugly head:”
Our reading: Hallucinations frequently occur in AI agents.
No horizon Evidence as of
Limitation: No standard rate measure; 'frequently' is judged from repeated documented cases.
Assessment: Can’t be checkedSaid
Forecast
“the “demo to reality” issue they mention is exactly what we should expect — probably for a years to come:”
Our reading: The gap between AI agent demos and real-world performance will probably persist for years to come.
No horizon
Qualifier: probably
Assessment: SupportedSaid
Causal claim
“As a consequence of their superficial understanding of what they’re being asked, current agents are grossly vulnerable to cyberattacks.”
Our reading: Current AI agents are grossly vulnerable to cyberattacks because of their superficial understanding of their tasks.
No horizon Evidence as of
Limitation: No study in the packet tests 'superficial understanding' as a cause directly. The causal link rests on how successful attacks worked and on one expert's explanation.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“An important, publicly-visible system that can be beat in .001% of the times it is attacked is a mess.)”
Our reading: An important, publicly visible system that can be beaten in .001% of the times it is attacked is a mess.
No horizon
Scope: important, publicly-visible system
Assessment: Can’t be checkedSaid
Observation
“What drives LLMs (and what drives the current batch of agents) is mimicry, and not what Ernest Davis and I called in 2019 deep understanding.”
Our reading: LLMs and current agents are driven by mimicry rather than deep understanding.
No horizon
Assessment: Can’t be checkedSaid
Observation
“Current systems can mimic the kinds of words people use in completing tasks, often in contextually relevant ways, but that doesn’t really mean that they understand the things that they are doing; they have no concept of what it means to delete a database or make up a fictitious calendar entry.”
Our reading: Current AI systems do not understand what they are doing and have no concept of actions like deleting a database.
No horizon
Assessment: SupportedSaid
Causal claim
“In fundamentally unreliable systems like LLMs, that means multiple chances for error. Soon or later, those errors catch up. That can and sometimes does lead to catastrophe.”
Our reading: Because LLMs are fundamentally unreliable, multi-step agentic tasks accumulate errors that sometimes lead to catastrophe.
No horizon Evidence as of
Limitation: The evidence is incident reports, not a measured rate of how often multi-step errors cause serious damage.
Assessment: Can’t be checkedSaid
Forecast
“But I seriously doubt that LLMs will ever yield the substrate we need.”
Our reading: LLMs are unlikely ever to provide the substrate needed for working AI agents.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“Someday, I predict, the failure of current agents to produce much of lasting value will be seen as a poignant reflection of a massive mistake, intellectually and economically speaking: the mistake of investing almost solely in LLMs, as a kind of hopeful shortcut.”
Our reading: Someday, current agents' failure to produce much of lasting value will be seen as a poignant reflection of a massive intellectual and economic mistake: investing almost solely in LLMs as a kind of hopeful shortcut.
No horizon
Qualifier: someday
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“In a rational world, generative AI would be penalized for failing to meet expectations around agents; in the actual world, investors and big tech companies are continuing to pour on the gas.”
Our reading: Generative AI should be penalized for failing to meet agent expectations, yet investors and big tech keep increasing investment.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Yet more good money is being poured after bad. Staggeringly large amounts of money.”
Our reading: Yet more good money, in staggeringly large amounts, is being poured after bad into generative AI.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Venture capitalists style themselves as brave funders of innovation, but in the current climate, they are anything but.”
Our reading: Venture capitalists style themselves as brave funders of innovation, but in the current climate they are anything but brave funders of innovation.
No horizon
Qualifier: in the current climate
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Perhaps after enough failures of AI agents, which shouldn’t be trusted with your calendar let alone your credit cards, reality may finally start settle in.”
Our reading: AI agents should not be trusted with your calendar, let alone your credit cards.
No horizon
Assessment: Can’t be checkedSaid
Conditional forecast
“Perhaps after enough failures of AI agents, which shouldn’t be trusted with your calendar let alone your credit cards, reality may finally start settle in.”
Our reading: Perhaps, after enough failures of AI agents, reality may finally start to settle in.
No horizon
Only if: after enough failures of AI agents
Held from this post (9)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The criteria are ambiguous. Our reading, not assessed: AI agents will not go away and will eventually be among the biggest time-savers humanity has known.
- The record does not meet the current rules. Our reading, not assessed: Despite close to a trillion dollars invested, the LLM approach has failed to yield systems that reliably handle calendars or bank accounts, let alone PhD-level agents.
- The criteria are ambiguous. Our reading, not assessed: AI coding agents seem to be incurring large technical debt by writing lots of hard-to-debug copypasta code.
- The criteria are ambiguous. Our reading, not assessed: Neurosymbolic AI and other alternatives are wildly underfunded, receiving perhaps 1% or less of total AI investment.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: Pure scaling is not getting us to AGI, and returns to scaling are diminishing.
- The criteria are ambiguous. Our reading, not assessed: Reliable agents require robust, trustworthy AI, and LLMs are not getting us there.
- The two reviews reached different assessments. Our reading, not assessed: Marcus says he just doesn't see how AI agents can work out without neurosymbolic AI that is more deeply integrated in systems as a whole, with rich world models as a central component.
- Attribution is uncertain