“So far most uses of LLMs have been for testing, not production-level systems.”
Our reading, the sentence that was checked:
Most uses of LLMs so far have been for testing rather than production systems.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, When Will the GenAI Bubble Burst?, paragraph 16.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-when-will-the-genai-bubble-burst-4518bf-a1
We recorded 19 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-28. The source may have been edited after 2024-03-31. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- So far
- most uses of LLMs
- for testing
- not production-level systems
Our interpretation
- The surrounding discussion concerns enterprise adoption, so I read uses as enterprise LLM applications or projects.
- Most means more than half.
- Testing includes experiments, proofs of concept and pilots; production means live operational use.
- So far describes activity up to March 31, 2024. No starting date is supplied, so the start remains unspecified.
- Respondent counts, organization counts, project counts and request volumes are different denominators and cannot be substituted without justification.
Counts as supported if: Representative evidence establishes that more than half of enterprise LLM applications attempted by the post date were testing rather than production systems.
Counts as contradicted if: Representative evidence with the same denominator establishes that half or fewer were testing.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
The O’Reilly survey supports the narrower observation that many respondents' companies were still experimenting. It does not measure the share of all LLM applications that were tests. Klarna demonstrates substantial production use, but cannot establish the opposite majority. The available evidence does not settle Marcus's quantitative claim.
Limitations
- Survey respondents are not interchangeable with projects or usage volume.
- Production examples cannot establish an industry-wide proportion.
- The principal pre-post survey predates the statement by several months.
Evidence (2 items)
Context only: O’Reilly,
18% of the respondents
Why it matters here: This is consistent with widespread experimentation in the September survey. It measures respondents' companies and generative AI broadly, not the share of all LLM projects that remained tests through March.
- Publisher
- O’Reilly
- Kind of source
- Independent measurement
- Describes events of
- Public from
Context only: Klarna Bank AB, distributed by PR Newswire,
2.3 million conversations
Why it matters here: Klarna reported a live customer-service application before the post. One substantial deployment does not determine the industry-wide majority, but shows why request volumes and project counts could yield different answers.
- Where
- Opening results list
- Publisher
- Klarna Bank AB, distributed by PR Newswire
- Kind of source
- Company disclosure: the company describing itself, not an independent finding
- Describes events of
- Public from
What would settle it
A representative deployment census or survey measuring the testing-versus-production share of enterprise LLM projects through March 31, 2024.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, When Will the GenAI Bubble Burst?
So far most uses of LLMs have been for testing, not production-level systems.
- Where
- paragraph 16
- Published
- 2024-03-31
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-when-will-the-genai-bubble-burst-4518bf-a1: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-when-will-the-genai-bubble-burst-4518bf-a1.