Humans versus Machines: The Hallucination Edition

Claims from this post (7)

In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.

1. Assessment: SupportedSaid 2024-04-21

   “The very process that LLMs use to generalize also creates hallucinations.”

   What we tested: The same process LLMs use to generalize is what creates their hallucinations.

   What happened: Hallucination follows from statistical prediction itself, even with ideal data. That matches the claim that the generalizing process creates hallucinations. (arXiv / STOC 2024 (Kalai and Vempala), 2023-11-24)

2. Assessment: MixedSaid 2024-04-21

   “Instead, everything that LLMs say -- true or false -- comes from the same process of statistically reconstructing what words are likely in some context. They NEVER fact-check what they say.”

   What we tested: All LLM output, true or false, comes from the same statistical word-reconstruction process, and LLMs never fact-check what they say.

   What held: This supports the first part. The leading LLM generates text by predicting likely next tokens. The report also notes post-training, which shapes but does not replace that process. (OpenAI (arXiv), 2023-03-15)

   What did not: From September 2023, Google's deployed LLM product, at the user's request, checked its own answers against web content. This contradicts 'NEVER fact-check' as applied to LLM products. (Google, 2023-09-19)

3. Assessment: Can’t be checkedSaid 2024-04-21

   “But even with perfect data, the stochastic reconstructive process would still produce some errors.”

   What we tested: Even with perfect data, the stochastic reconstructive process LLMs use would still produce some errors.

4. Assessment: MixedSaid 2024-04-21

   “Instead, LLM "hallucinations" arise, regularly, because (a) they literally don't know the difference between truth and falsehood, (b) they don't have reliably reasoning processes to guarantee that their inferences are correct and (c) they are incapable of fact-checking their own work.”

   What we tested: LLM hallucinations arise regularly because LLMs literally don't know the difference between truth and falsehood, don't have reasoning processes reliable enough to guarantee their inferences are correct, and are incapable of fact-checking their own work.

   What held: In the reported intrinsic-correction experiments, models had to judge their answers without ground-truth feedback, and accuracy fell. This supports unreliable reasoning and self-correction in those settings, not universal incapacity or the full explanation of factual hallucinations. (arXiv; Huang et al., 2024)

   What did not: The model generated verification questions, answered them using the model itself without external tools, and revised its response. The reported improvements contradict categorical inability under the stated behavioral reading, while leaving substantial errors. (arXiv; Dhuliawala et al., 2023)

5. Assessment: Can’t be checkedSaid 2024-04-21

   “Rather, the goal of AGI should be to build machines that can reliably reason and plan about a wide swathe of the world.”

   What we tested: The goal of AGI should be machines that reliably reason and plan about a wide swathe of the world, not recreating humans.

6. Assessment: Can’t be checkedSaid 2024-04-21

   “The field of AI will eventually do better, but probably with an AI that is structured differently, in which facts are first-class citizens, rather than something you hope you might get for free with enough data.”

   What we tested: AI will eventually do better on hallucinations, probably via differently structured systems that treat facts as first-class rather than via more data.

7. Assessment: Can’t be checkedSaid 2024-04-21

   “TL;DR: Don't console yourself with making something that superficially looks like human errors, if you aspire to AGI.”

   What we tested: Those aiming for AGI should not take comfort in LLM errors superficially resembling human errors.

Held from this post (2)

These records did not pass every check, so nothing about them is published. Held is not a finding.

• The criteria are ambiguous. Our reading, not assessed: LLM errors result from a different underlying process than human errors: because LLMs lack intentions, egos, and financial interests, they do not lie, and they also do not take drugs and do not have emotional states.
• The two reviews read the claim differently