“The fourth thing to realize is that we, and by this I mean humanity as a whole, still don’t really have an adequate methodology for building complex cognitive systems.”
Our reading, the sentence that was checked:
Humanity lacks an adequate methodology for building complex cognitive systems.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Dear Elon Musk, here are five things you might want to consider about AGI, paragraph 27.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-dear-elon-musk-here-are-five-things-056704-a1
We recorded 21 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2022-05-31. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- humanity as a whole
- still don’t really have
- an adequate methodology
- building complex cognitive systems
Our interpretation
- An adequate methodology means a repeatable engineering approach for building and modifying integrated cognitive systems with dependable behavior and manageable regressions.
- Adequacy does not require eliminating every bug. Conversely, a successful isolated model does not establish a generally adequate integration methodology.
- The context concerns perception, reasoning, planning and action working together, including autonomous vehicles.
- The current observation window is calendar 2022 through May 31 because the research field has no regular reporting cycle; earlier methods are context.
Counts as supported if: Broad contemporary evidence shows that available engineering approaches cannot reliably manage integration and modification of complex cognitive systems across relevant applications.
Counts as contradicted if: A publicly documented, repeatable methodology had demonstrated dependable construction and modification of such systems across relevant applications by May 2022.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The industry study corroborates the specific integration and regression problems Marcus describes. However, concrete testing methods and integrated systems also existed. Persistent failures do not establish that humanity lacked an adequate methodology, just as a successful demonstration does not establish a generally dependable one. The retrieved evidence cannot settle the field-wide claim at the stated level of adequacy.
Limitations
- The empirical study covers a sample of practitioners and systems.
- The evidence mixes production engineering practices with research demonstrations.
- Publication dates do not identify the exact dates of all underlying experiments or interviews.
Evidence (3 items)
Supports the claim: Microsoft Research; ICSE 2022,
regression on model performance.
Why it matters here: The study reports industry integration and regression problems matching Marcus's concern, while also identifying useful practices. The page gives publication month only; available_at uses that month's final day as a conservative availability bound.
- Where
- Abstract, finding 2
- Publisher
- Microsoft Research; ICSE 2022
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
Context only: arXiv; Scott Reed and colleagues,
multi-embodiment generalist policy.
Why it matters here: Gato demonstrates integration across several kinds of tasks and embodiments before the post. That is relevant counterevidence to an inability-to-build reading, but it does not demonstrate dependable maintenance or regression control under the contract.
- Where
- Abstract
- Publisher
- arXiv; Scott Reed and colleagues
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: Google Research; Proceedings of IEEE Big Data,
technical debt
Why it matters here: The paper proposes 28 tests and monitoring needs based on production experience, establishing that concrete engineering methods existed. It predates the target period, and the page supplies only a publication year; available_at is a conservative year-end bound.
- Where
- Abstract
- Publisher
- Google Research; Proceedings of IEEE Big Data
- Kind of source
- Peer-reviewed paper
- Describes events of
- Public from
What would settle it
A defined reliability standard and comparative evidence showing whether contemporary methods could repeatedly build, modify and maintain representative integrated cognitive systems to that standard.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Dear Elon Musk, here are five things you might want to consider about AGI
The fourth thing to realize is that we, and by this I mean humanity as a whole, still don’t really have an adequate methodology for building complex cognitive systems.
- Where
- paragraph 27
- Published
- 2022-05-31
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-dear-elon-musk-here-are-five-things-056704-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-dear-elon-musk-here-are-five-things-056704-a1.