“agents as a concept are fucked because large language models are inherently unreliable”
Our reading, the sentence that was checked:
AI agents are fundamentally unworkable because LLMs are inherently unreliable and break down on multi-step tasks.
Assessment: Unresolved Evidence as of
Where and when Ed Zitron wrote it
Where's Your Ed At, Make Fun Of Them, paragraph 147.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment wyea-make-fun-of-them-3ecb07-2-a1
We recorded 33 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-06-30. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Causal claim
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- agents as a concept are fucked
- large language models are inherently unreliable
- agents began to break down when given multi-step tasks
Our interpretation
- The ordinary reading is that LLM unreliability prevents agents from dependably completing useful multistep workflows.
- The assessment concerns the normalized claim about agents. The separate assertion about Apple's reasoning paper is outside its scope.
- Failures in particular tests support a reliability concern but do not alone establish that agents are fundamentally unworkable.
- Ordinary safeguards, verification and error recovery count as parts of an agent system.
Counts as supported if: Representative evaluations establish that LLM unreliability prevents useful, dependable multistep automation, including when ordinary safeguards and error recovery are used.
Counts as contradicted if: Reproducible evaluations establish useful multistep agent workflows that meet practical reliability requirements, showing that underlying model errors need not make the system unworkable.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Does the evidence support the cause?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The reliability concern has substantial support. Salesforce found marked deterioration when agents needed clarification, and METR documented difficulties with sustained work. Neither establishes that agents are fundamentally unworkable. Salesforce also found stronger workflow performance, and later architectural experiments report substantial improvements. Those counterexamples weaken the causal generalization but do not settle dependable real-world operation across the claim's unspecified scope.
Limitations
- Multi-turn interaction is not synonymous with multistep execution.
- Benchmark results do not directly establish deployment reliability or economic usefulness.
- Later evidence bears on the claimed inherent limitation, not on what was available in June 2025.
Evidence (3 items)
Supports the claim: Salesforce AI Research / arXiv,
substantial decrease
Why it matters here: Supports the narrower reliability concern: overall success fell from approximately 58% to 35% when agents needed conversational clarification. The same paper reports over 83% success for some workflow-execution tasks, so its findings do not establish universal unworkability.
- Publisher
- Salesforce AI Research / arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Supports the claim: METR,
remote executive assistance.
Why it matters here: Supports contemporary concerns about sustained autonomous work. The study also finds high success on short tasks and improving capabilities; it does not establish an inherent ceiling.
- Publisher
- METR
- Kind of source
- Independent measurement
- Describes events of
- Public from
Context only: Leni Inc. / arXiv,
structure is.
Why it matters here: The developer reports 91.25% success on SpreadsheetBench and 75.2% single-attempt success on GAIA, with gains from system architecture and specialized models. This challenges the inference from model fallibility to system impossibility, but independent replication and deployment-wide reliability are not established.
- Publisher
- Leni Inc. / arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Independent, repeated evaluations of realistic workflows with explicit reliability requirements, intervention counts and controlled comparisons of error-recovery designs.
Where it was said (1)
Said once in the sources this register covers.
: Where's Your Ed At, Make Fun Of Them
agents as a concept are fucked because large language models are inherently unreliable
- Where
- paragraph 147
- Published
- 2025-06-30
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment wyea-make-fun-of-them-3ecb07-2-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.