Hello, Multimodal Hallucinations
We recorded 11 claims from this post. 7 were checked against evidence. 4 can’t be checked. 1 more is held. Read the post at its source.
Claims from this post (11)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
“ChatGPT went utterly wild when Ernie pushed on the last one, omitting some that are there and and confabulating a standalone island in the middle of the kitchen that simply isn’t there:”
What we tested: When questioned about the kitchen image, ChatGPT's vision model left out drawers that are there and made up a kitchen island that is not there.
Assessment: SupportedSaid
“As is by now widely known, they are a hallmark of systems that generalize based on word distribution.”
What we tested: As is by now widely known, hallucinations are a hallmark of systems that generalize based on word distribution.
What happened: The survey identifies hallucination as a substantial reliability problem and organizes existing detection, explanation and mitigation research. This supports broad recognition of a recurring failure, without supplying a universal frequency. (arXiv; Yue Zhang and colleagues, )
Assessment: Can’t be checkedSaid
“GPT-v (the visual version of GPT) can use words like refrigerator and counter, but that doesn’t mean it actually knows what these things mean..”
What we tested: GPT-4V can use words like refrigerator and counter, but that does not show it knows what they mean.
Assessment: SupportedSaid
“As we can see, multimodal training is not magically making hallucinations go away.”
What we tested: Multimodal training is not making hallucinations go away.
What happened: The release assessment reports hallucinations in GPT-4V itself. Its Be My Eyes discussion also reports reductions in hallucination frequency and severity during testing, supporting improvement without elimination. (OpenAI, )
Assessment: Can’t be checkedSaid
“The old systems were lost in a sea of correlations between words; the new system are lost in a sea of correlations between words and pixels.”
What we tested: Multimodal models, like text-only ones before them, depend on correlations (now between words and pixels) instead of understanding.
Assessment: Can’t be checkedSaid
“It may get fixed, with a band-aid of some sort or another, but we are confident a zillion other pictures like it will probably yield similar errors.”
What we tested: Even if this specific kitchen example gets patched, many other similar pictures will probably still produce similar errors in multimodal LLMs.
Assessment: UnresolvedSaid
“To our knowledge, no LLM-based system can yet reliably do what SHRLDU tried to do then:”
What we tested: As far as the authors know, no LLM-based system can yet reliably do the blocks-world modeling that SHRDLU attempted in the 1970s.
Assessment: Can’t be checkedSaid
“The whole point of Rebooting AI was that the AI of 2018 didn’t actually understand how the world works, and that we couldn’t trust AI that didn’t. We stand fully by what we said.”
What we tested: The AI of 2018 did not actually understand how the world works, AI that does not understand how the world works cannot be trusted, and the authors fully stand by both points from Rebooting AI.
Assessment: SupportedSaid
“To be sure, large language models are, in a lots of ways, an incredible advance over the AI of the 1970s. They can engage in a much broader range of tasks, and generally they need less hand-tinkering (Though they still need some – that’s what reinforcement learning with human feedback is really about.)”
What we tested: In a lot of ways, large language models are an incredible advance over the AI of the 1970s: they can engage in a much broader range of tasks and generally need less hand-tinkering, though they still need some, which is what reinforcement learning with human feedback is really about.
What happened: The report describes instruction-based generalization and evaluations across diverse held-out tasks. Compared with the bounded historical setting, this supports broader reuse with less task-specific programming, while its training-method discussion still documents substantial human design work. (Google Research, )
Assessment: UnresolvedSaid
“And no matter what investors and CEOs might tell you, hallucinations will remain inevitable.”
What we tested: Hallucinations in large language models will remain inevitable.
Assessment: UnresolvedSaid
“But in other ways, large language models are a profound step backwards; they lack stable models of the world, which means that they are poor at planning, and unable to reason reliably; everything becomes hit or miss..”
What we tested: LLMs lack stable world models, which makes them poor at planning and unable to reason reliably.
Held from this post (1)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews reached different assessments. Our reading, not assessed: Marcus offers to take Reid Hoffman's bet for $100,000, betting against Hoffman's claim that hallucinations can be brought down to the human-expert rate within months.