Shame on Google, twice
We recorded 2 claims from this post. 1 was checked against evidence. 1 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (2)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
“Me, December 2022: The problem with LLMs is that they hallucinate, and their errors can be hard to catch.”
What we tested: As of December 2022, LLMs hallucinated and their errors could be hard to catch.
What happened: Kubacka describes checking plausible scientific citations and discovering nonexistent articles, mismatched identifiers, and invented physics. She reports that an explanation in her own field initially made her reconsider whether the invented phenomenon existed, directly supporting the possibility of difficult-to-detect errors. (Teresa Kubacka, Lookalikes and Meanders, )
Assessment: Can’t be checkedSaid
“So much for exponential progress.”
What we tested: Google repeating the same kind of LLM error two years apart mocks claims of exponential progress in AI.
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: In February 2025 Google again showcased an LLM output that contained a hallucinated error, repeating its February 2023 mistake.