“Eleven years later, comprehension is still lacking (see any of my recent essays here) and very few if any jobs have actually been replaced by AI.”
Our reading, the sentence that was checked:
As of October 2022, AI still lacks comprehension, and very few if any jobs have actually been replaced by AI.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Three baffling claims about AI and machine learning in four days, statistical errors in top journals, and claims from Yann LeCun that you should not believe., paragraph 4.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-too-much-benefit-of-the-doubt-723cae-a1
We recorded 13 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-10-16. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- Eleven years later
- comprehension is still lacking
- very few if any jobs
- actually been replaced by AI
Our interpretation
- The statement combines two propositions: AI still lacked robust general comprehension, and actual replacement of human jobs by AI remained rare.
- In context, comprehension means understanding language and situations across unfamiliar settings, rather than succeeding on an isolated language benchmark.
- Jobs means human employment positions. Automating tasks, reducing hiring, and eliminating an entire occupation are different outcomes.
- I read very few as a small share of employment, without inventing a numerical cutoff or restricting the claim to one country.
- The target interval follows the explicit comparison with 2011. Job replacement is cumulative through the post; comprehension concerns the endpoint. Earlier comprehension tests are context rather than direct measurements of the October endpoint.
Counts as supported if: Representative evidence shows that actual AI-caused job displacement remained small through October 16, 2022, and broad evaluations of systems existing then show substantial failures of general comprehension.
Counts as contradicted if: Reliable evidence establishes substantial actual job replacement attributable to AI during the interval, or an existing system demonstrates robust general comprehension across unfamiliar settings. Isolated job losses do not alone contradict very few.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The records are consistent with limited aggregate employment effects and important comprehension weaknesses. They do not settle both parts of the claim. The employment study measures hiring and aggregate changes, not all jobs actually eliminated by AI. The language experiments demonstrate specific weaknesses, not an absence of comprehension across all October 2022 systems. Reduced hiring also prevents treating the employment evidence as showing no substitution.
Limitations
- Very few has no supplied numerical threshold.
- The strongest employment evidence has narrower geographic and temporal coverage than the claim.
- Benchmark failures do not establish a universal absence of understanding.
Evidence (2 items)
Context only: National Bureau of Economic Research,
reducing hiring in non-AI positions
Why it matters here: The study reports reduced hiring at exposed establishments and no detectable aggregate employment relationship in its earlier US data. Neither finding counts gross jobs eliminated worldwide through October 2022; available_at conservatively uses the last day of the stated February 2022 revision month.
- Publisher
- National Bureau of Economic Research
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: arXiv,
prompting
Why it matters here: The experiments show that success on one formulation of a causal-language task need not transfer to another. This supports concern about robustness, but a June evaluation of selected models does not establish an October-wide absence of comprehension.
- Publisher
- arXiv
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
A representative historical estimate of actual AI-caused job displacement through the post, plus broad evaluations of the systems available then under an explicit, ordinary comprehension standard.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Three baffling claims about AI and machine learning in four days, statistical errors in top journals, and claims from Yann LeCun that you should not believe.
Eleven years later, comprehension is still lacking (see any of my recent essays here) and very few if any jobs have actually been replaced by AI.
- Where
- paragraph 4
- Published
- 2022-10-16
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-too-much-benefit-of-the-doubt-723cae-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-too-much-benefit-of-the-doubt-723cae-a1.