“it’s often easy to get a solution that is 80% correct, and extremely difficult to get the last 20%. Driverless cars have been like that for decades.”
Our reading, the sentence that was checked:
In AI it is often easy to reach 80% correctness and extremely difficult to get the last 20%, as with driverless cars for decades.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Decoding (and debunking) Hard Fork’s Kevin Roose, paragraph 16.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-decoding-and-debunking-hard-forks-69efcf-a1
We recorded 16 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2025-03-02. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- often easy
- 80% correct
- extremely difficult
- the last 20%
- Driverless cars have been like that for decades
Our interpretation
- The ordinary reading is a development heuristic: initial useful functionality often comes more easily than eliminating remaining failures.
- The 80/20 wording is approximate, not an asserted universal measured ratio.
- The driving example requires historical evidence spanning decades; the broader statement requires evidence beyond one industry's difficult cases.
- The target is the historical development experience through the post. The speaker gives no exact beginning.
- Successful deployment within a restricted operating area does not imply that every remaining driving problem has been solved.
Counts as supported if: Representative development histories or reliability studies show repeatedly disproportionate effort to eliminate residual failures after comparatively easy initial progress, including a decades-long driving history.
Counts as contradicted if: Representative evidence shows that residual failures are generally no harder to remove than initial functionality is to achieve, or decisively disproves the asserted driving history.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The driving record supports substantial early capability followed by persistent difficulty with rare safety-critical cases. Waymo's deployment also shows major practical progress. These records do not establish the broader claim that initial AI development is often easy and the remaining work extremely difficult, using comparable measures of effort and correctness. The heuristic is plausible but not settled by the evidence collected.
Limitations
- Distance automated is not equivalent to correctness across all driving situations.
- The research comment explains a technical difficulty rather than surveying development effort.
- Deployment success neither proves complete reliability nor refutes disproportionate difficulty with residual cases.
Evidence (3 items)
Supports the claim: Carnegie Mellon University School of Computer Science,
an epic journey
Why it matters here: CMU's institutional account reports computer driving for more than 98% of the trip, documenting substantial early functionality decades before the post. It does not measure initial engineering effort or establish an overall correctness percentage.
- Publisher
- Carnegie Mellon University School of Computer Science
- Kind of source
- News report
- Describes events of
- Public from
Supports the claim: Nature Communications,
hinders the ability of deep learning models
Why it matters here: The research comment explains how rare safety-critical events and many interacting variables make further safety improvements difficult. It supports the residual-failure difficulty component, but does not measure a universal 80/20 split or show that initial development was easy.
- Publisher
- Nature Communications
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
Context only: Waymo,
over 4 million fully autonomous rides
Why it matters here: Waymo's reported deployment is material counterevidence to any reading that driverless technology had made little practical progress. It does not settle relative development effort or performance outside its operating areas.
- Publisher
- Waymo
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
What would settle it
Comparative project histories measuring effort against reliability gains across several AI domains, with stable task scope, plus equivalent longitudinal evidence for autonomous driving.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Decoding (and debunking) Hard Fork’s Kevin Roose
it’s often easy to get a solution that is 80% correct, and extremely difficult to get the last 20%. Driverless cars have been like that for decades.
- Where
- paragraph 16
- Published
- 2025-03-02
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-decoding-and-debunking-hard-forks-69efcf-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-decoding-and-debunking-hard-forks-69efcf-a1.