“A new study from researchers from Stanford, MIT CSAIL, Carnegie Mellon, ITU Copenhagen, and NVIDIA and Elloe AI Labs, examining 847 autonomous agent deployments drawn from healthcare, finance, customer service and code-generation, showed that 91% were vulnerable to subtle but dangerous tool-chaining attacks (see diagram below),”
Our reading, the sentence that was checked:
A new study found 91% of 847 autonomous agent deployments were vulnerable to tool-chaining attacks.
Assessment: Supported Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Breaking: Autonomous Agents are a Shitshow, paragraph 3.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-breaking-autonomous-agents-are-a-a07a65-a2
We recorded 4 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2026-05-05. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
Conditions
- per new multi-institution study (qualifier, stated by the speaker)
From the speaker
- new study
- Stanford, MIT CSAIL, Carnegie Mellon, ITU Copenhagen, and NVIDIA and Elloe AI Labs
- 847 autonomous agent deployments
- healthcare, finance, customer service and code-generation
- 91% were vulnerable to subtle but dangerous tool-chaining attacks
Our interpretation
- This claim reports what a study found. It is not a claim that 91% of all real-world agents are vulnerable. It holds if the cited study reported that about 91% of 847 deployments in those sectors were susceptible to tool-chaining attacks.
- No link to the paper was given and the primary text could not be found. A reputable report on the paper's findings, especially one quoting its author, counts as evidence of what the paper reported.
Counts as supported if: The study, or a reliable report of it, says that it examined 847 agent deployments in healthcare, finance, customer service and code generation, and that 91% were susceptible to tool-chaining attacks.
Counts as contradicted if: The study, or a reliable report of it, gives a materially different sample size or rate, or reports no such tool-chaining figure.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Supported
The dated evidence matches what the claim committed to, under the criteria frozen before grading.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
This claim reports what a study found. GovInfoSecurity (ISMG), on 2026-05-08, said researchers from Elloe AI Research Lab, Stanford, MIT, Carnegie Mellon, IT University of Copenhagen and Nvidia analyzed 847 agent deployments across healthcare, finance, customer service and software development (e1). The same report said 91% of tested deployments were susceptible to tool-chaining vulnerabilities (e2). Wallstreetcn, on 2026-05-06, gave the same figures: 91% of 847 evaluated production deployments had tool-chaining vulnerabilities (e3). The paper itself could not be found, so this assessment rests on news reports of its findings, which the contract accepts. It checks whether Marcus reported the study's findings accurately. It does not check whether the study's methods are sound.
Limitations
- The primary paper was not found online. The assessment rests on news reports.
- The study's methods, and how it defined 'vulnerable', could not be checked.
- It is unclear whether the deployments were tested live or in a controlled setting.
- The Wallstreetcn report cites Marcus and may not be fully independent of his post.
- It is unconfirmed whether the Elloe 'Aegis' paper listed on 2026-02-20 is this study.
Evidence (3 items)
Supports the claim: GovInfoSecurity (ISMG),
Researchers from Elloe AI Research Lab, Stanford University, Massachusetts Institute of Technology, Carnegie Mellon University, IT University of Copenhagen and Nvidia analyzed 847 agent deployments across healthcare, finance, customer service and software development.
Why it matters here: Confirms the study's institutions, its sample of 847 deployments and its four sectors, as Marcus described them.
- Where
- paragraph 3
- Publisher
- GovInfoSecurity (ISMG)
- Kind of source
- News report
- Describes events of
- Public from
Supports the claim: GovInfoSecurity (ISMG),
The succession of events creates tool-chaining vulnerabilities: sequences of individually permitted actions that collectively produce outcomes no single access control would allow. Ninety-one percent of tested deployments were susceptible.
Why it matters here: Reports the study's finding that 91% of tested deployments were susceptible to tool-chaining attacks, which matches the claim.
- Where
- paragraph 4
- Publisher
- GovInfoSecurity (ISMG)
- Kind of source
- News report
- Describes events of
- Public from
Supports the claim: Wallstreetcn,
在所评估的847个自主智能体生产部署中，91%存在工具链攻击漏洞
Why it matters here: A second report gives the same figures: of 847 production deployments evaluated, 91% had tool-chaining vulnerabilities. It also adds a sector breakdown, but it may partly draw on Marcus's post.
- Where
- paragraph 3
- Publisher
- Wallstreetcn
- Kind of source
- News report
- Describes events of
- Public from
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Breaking: Autonomous Agents are a Shitshow
A new study from researchers from Stanford, MIT CSAIL, Carnegie Mellon, ITU Copenhagen, and NVIDIA and Elloe AI Labs, examining 847 autonomous agent deployments drawn from healthcare, finance, customer service and code-generation, showed that 91% were vulnerable to subtle but dangerous tool-chaining attacks (see diagram below),
- Where
- paragraph 3
- Published
- 2026-05-05
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-breaking-autonomous-agents-are-a-a07a65-a2: Current. Outcome: Supported. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-breaking-autonomous-agents-are-a-a07a65-a2.