“In practice, those mistakes and errors have greatly limited the utility agents like ChatGPT agent.”
Our reading, the sentence that was checked:
Mistakes and errors have greatly limited the utility of agents like ChatGPT agent.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, AI Agents have, so far, mostly been a dud, paragraph 16.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-ai-agents-have-so-far-mostly-been-0c896c-a1
We recorded 21 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-08-03. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Causal claim
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- In practice
- those mistakes and errors
- have greatly limited the utility
- agents like ChatGPT agent
Our interpretation
- The plain reading is that errors substantially reduced usable task completion or imposed substantial checking and correction work.
- The claim does not require agents to be useless, or errors to be the sole or largest limitation.
- ChatGPT agent is the named example. Assess its introduction-to-publication period, July 17 through August 3, 2025; evidence about earlier agents is background.
- Distinguish model mistakes from deliberate approval requirements, access restrictions, outages and slow execution.
Counts as supported if: Task evaluations or deployment records directly connect errors to substantial loss of usable results or substantial correction burdens across ordinary uses.
Counts as contradicted if: Comparable records show that errors caused little practical loss of utility, with usable completion and correction burdens largely unaffected.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Does the evidence support the cause?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Firsthand reports show that mistakes spoiled some outputs and required correction. Other contemporaneous tests found substantial practical benefits. These findings can coexist: useful agents can still lose considerable utility to errors. The available records establish examples of the mechanism, but do not settle the broad magnitude claimed.
Limitations
- The causal evidence consists of small, selected task demonstrations.
- No retrieved study estimates the total utility lost to errors during the target period.
- Positive examples are counterweight, not a direct contradiction of a substantial limitation.
Evidence (2 items)
Supports the claim: Thomas Manandhar-Richardson,
not successful.
Why it matters here: The firsthand account links failed execution and inaccurate outputs to unusable or repair-needing results in several tasks. It supports the proposed mechanism, but does not quantify the overall utility lost because of errors.
- Publisher
- Thomas Manandhar-Richardson
- Kind of source
- Demonstration
- Describes events of
- Public from
Context only: Tom's Guide,
real, time-consuming tasks
Why it matters here: This firsthand review found practical benefits in shopping, planning and administrative tasks. It is material positive evidence, but usefulness in selected tasks does not measure or disprove the loss attributable to errors.
- Publisher
- Tom's Guide
- Kind of source
- News report
- Describes events of
- Public from
What would settle it
Contemporaneous task or deployment records measuring successful usable outputs, checking and repair time, and failure causes across representative uses of ChatGPT agent and comparable agents.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, AI Agents have, so far, mostly been a dud
In practice, those mistakes and errors have greatly limited the utility agents like ChatGPT agent.
- Where
- paragraph 16
- Published
- 2025-08-03
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-ai-agents-have-so-far-mostly-been-0c896c-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-ai-agents-have-so-far-mostly-been-0c896c-a1.