Why DO large language models hallucinate?
We recorded 9 claims from this post. 4 were checked against evidence. 5 can’t be checked. 1 more is held. Read the post at its source.
Claims from this post (9)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Even the best performance (15% hallucination rate) is, relative to an open-book exam with sources supplied, pathetic.”
Our reading: Even the best performance in the cited study, a 15% hallucination rate, is pathetic for an open-book exam where the sources were supplied.
No horizon
Scope: cited benchmark of 60 questions with CNN source articles supplied
Assessment: Can’t be checkedSaid
Forecast
“But LLMs still cannot - and on their own may never be able to — reliably do even things that basic.”
Our reading: LLMs on their own may never be able to reliably check their output against basic sources.
No horizon
Qualifier: on their own
Assessment: UnresolvedSaid
Observation
“They don’t have a database of records like any proper business would (which would be a strong basis to solve the problem); and they don’t have what people like Yann LeCun or Judea Pearl or I would call a world model.”
Our reading: LLMs don't have a database of records like any proper business would, and they don't have what people like Yann LeCun, Judea Pearl, or Marcus would call a world model.
No horizon Evidence as of
Limitation: One documented architecture cannot establish a claim about all LLMs.
Assessment: UnresolvedSaid
Causal claim
“(Those systems also can’t reliably stick to the rules of chess despite having them – and millions of games – in their database, a manifestation of the related problem of extracting statistical tendencies without every fully deriving and apprehending the correct abstractions).”
Our reading: LLMs can't reliably stick to the rules of chess despite having the rules and millions of games in their data, and this is a manifestation of the related problem of extracting statistical tendencies without ever fully deriving and grasping the correct abstractions.
No horizon Evidence as of
Limitation: Puzzle results, full-game results, and playing strength are not interchangeable.
Assessment: Can’t be checkedSaid
Observation
“LLMs have their uses – next-token prediction is great for a kind of fancy autocomplete for coding, for example — but it is a fallacy to think that because GenAI outputs sound human-like that their computations are human-like.”
Our reading: LLMs are useful e.g. for coding autocomplete, but their human-sounding outputs do not imply human-like computation.
No horizon
Assessment: Can’t be checkedSaid
Observation
“LLMs mimic the rough structure of human language, but 8 years and roughly half a trillion dollars after their introduction, they continue to lack a grasp of a reality.”
Our reading: After 8 years and roughly half a trillion dollars, LLMs still lack a grasp of reality.
No horizon
Assessment: ContradictedSaid
Causal claim
“And they have such a superficial understanding of their own output that they can’t begin to fact check it.”
Our reading: LLMs have such a superficial understanding of their own output that they can't even begin to fact-check it.
No horizon Evidence as of
Limitation: The verdict turns on the stronger inability claim. A claim of unreliable self-checking would receive a different assessment.
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We will eventually get to something better, but continuing to put all our eggs in the Henrietta’s LLM basket is absurd to the point of being delusional.”
Our reading: The field should not put all its eggs in the LLM basket; doing so is delusional.
No horizon
Assessment: UnresolvedSaid
Forecast
“We will eventually get to something better, but continuing to put all our eggs in the Henrietta’s LLM basket is absurd to the point of being delusional.”
Our reading: AI will eventually get to something better than LLMs.
No deadline stated Evidence as of
Limitation: The result depends on how broadly 'something better' is read.
Held from this post (1)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The record does not meet the current rules. Our reading, not assessed: LLMs cannot reliably stick to Wikipedia content despite having trained on it.