This Week in AI Doublespeak
We recorded 3 claims from this post. 2 were checked against evidence. 1 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (3)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: UnresolvedSaid
“if the definition of reasoning is to obtain valid conclusions from known facts, GPT-4 frequently falls short there, too.”
What we tested: GPT-4 frequently fails to reason, if reasoning means reaching valid conclusions from known facts.
Assessment: SupportedSaid
“True LLMs don’t (just) memorize, and true that their models make for lousy databases (for a database a hallucination is an outright fail)”
What we tested: OpenAI's LLMs make lousy databases because hallucinations amount to outright failures for a database.
What happened: OpenAI acknowledged factual fabrication in the released model and recommended checks appropriate to the use case. The same section reports improved factuality over GPT-3.5, but explicitly says the problem remains. (OpenAI, )
Assessment: Can’t be checkedSaid
“So-called “reasoning” by free association, even constrained by a giant database, isn’t really reasoning:”
What we tested: The free-association 'reasoning' LLMs do is not really reasoning, even when constrained by a giant database.
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.