“Large language models are without equal at generating misinformation, but they suck at detecting it.”
Our reading, the sentence that was checked:
LLMs are unmatched at generating misinformation but poor at detecting it.
Assessment: Unresolved Evidence as of
Where and when Gary Marcus wrote it
Marcus on AI, Bravo @ElonMusk for wanting Twitter to become the world’s “most accurate source of information”, paragraph 16.
- Said
- Source observed
- live page, full text, public
- Evidence as of
- the cutoff for this assessment
- Assessment recorded
- assessment marcus-bravo-elonmusk-for-wanting-twitter-3a095d-a2
We recorded 9 claims from this post. See every claim from this post.
The wording above is the version this site observed on 2026-10-29. The source may have been edited after 2022-11-09. This page does not claim the wording was captured then.
What the claim commits to
- Kind
- Observation
- Deadline, as the speaker wrote it
- None stated. This site does not add one.
- Deadline state when assessed
- No horizon
- Deadline state on 2026-10-29
- No horizon (the release’s reference date)
From the speaker
- Large language models
- without equal at generating misinformation
- they suck at detecting it
Our interpretation
- This describes capabilities at the time of the post, not later models.
- There is no regular reporting period for model capabilities. I use the current calendar year through the post, 2022-01-01 to 2022-11-09, and require evidence applicable to models available then.
- The ordinary reading is that LLMs were exceptionally effective, unmatched producers of convincing false content but unreliable detectors of false information.
- Detection means judging factual accuracy, not identifying whether text was written by a machine.
- Unmatched generation requires comparative evidence. Fluency or the production of some false answers alone does not establish superiority.
Counts as supported if: Representative comparisons applicable to the target period establish both superior misinformation generation relative to human and other automated alternatives and broadly unreliable factual detection by contemporary LLMs.
Counts as contradicted if: Comparable evidence applicable to the target period establishes that alternatives match or exceed LLM misinformation generation, or that contemporary LLMs reliably detect misinformation across representative tasks. Narrow successes would qualify the claim without necessarily settling it.
Retrospective: these criteria were written after the outcome was public.
Current assessment
Was it true for the period described?
Outcome: Unresolved
The record was checked, and the evidence available by the cutoff does not settle it.
Scored on a plain reading. The speaker named no measure for this claim, so it was judged on the reading an informed reader would take, stated below under our interpretation. Both reviews had to find that reading fair.
- Evidence as of
- Deadline state when assessed
- No horizon
- Assessment recorded
- Policy version
- 2026-10-28.7
Rationale
Neither item settles the claim for January 1–November 9, 2022 [e1, e2]. The September 2021 TruthfulQA report measures false-answer generation, not misinformation detection or comparative effectiveness at producing misinformation [e1]. The January 2023 GPT-3 preprint reports 89% detection success on false tweets and 64% on accurate tweets, but the quoted experiment has not been dated to the target period [e2]. These results do not establish either unmatched misinformation generation or broadly poor detection by LLMs during that period [e1, e2].
Limitations
- No admitted comparison establishes unmatched misinformation generation during the target period.
- The detection evidence covers a limited tweet dataset, and its experimental timing is not established.
- TruthfulQA measures answer truthfulness rather than factual detection.
Evidence (2 items)
Context only: arXiv; Stephanie Lin, Jacob Hilton and Owain Evans,
truthful on 58% of questions
Why it matters here: This documents substantial false-answer generation in an earlier evaluation. It neither measures misinformation detection directly nor establishes unmatched misinformation production in 2022.
- Where
- Abstract
- Publisher
- arXiv; Stephanie Lin, Jacob Hilton and Owain Evans
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
Context only: arXiv; Giovanni Spitale, Nikola Biller-Andorno and Federico Germani,
performed slightly better
Why it matters here: The study reports compelling GPT-3 disinformation alongside meaningful detection ability: 89% success on false tweets, but only 64% on accurate tweets. January 2023 is the documented reporting period; the quoted experiment's timing was not established, so these results are contextual rather than decisive for November 2022.
- Publisher
- arXiv; Giovanni Spitale, Nikola Biller-Andorno and Federico Germani
- Kind of source
- Preprint, not peer reviewed
- Describes events of
- Public from
What would settle it
Evaluations tied to models and experiments from January 1–November 9, 2022, comparing misinformation credibility and production efficiency against human and automated alternatives, and testing factual detection across representative topics with human or established-system baselines and both missed falsehoods and false accusations.
Where it was said (1)
Said once in the sources this register covers.
: Marcus on AI, Bravo @ElonMusk for wanting Twitter to become the world’s “most accurate source of information”
Large language models are without equal at generating misinformation, but they suck at detecting it.
- Where
- paragraph 16
- Published
- 2022-11-09
- Source observed
- , live page
- Access
- full text, public
History
Assessment versions
- Assessment marcus-bravo-elonmusk-for-wanting-twitter-3a095d-a2: Current. Outcome: Unresolved. Evidence as of 2026-10-28. Public date not recorded.
Recorded events
Assessment publishedBy this site
First assessment published after both reviews and the final-text check passed.
About assessment marcus-bravo-elonmusk-for-wanting-twitter-3a095d-a2.