A Tale of Two Liars
We recorded 3 claims from this post. 2 were checked against evidence. 1 can’t be checked. 2 more are held. Read the post at its source.
Claims from this post (3)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
“GPT’s “lies” are also legendary, and prolific, but not really lies, because real lies intend to deceive, and GPT has no intent.”
What we tested: GPT produces many falsehoods, but they are not really lies because GPT has no intent to deceive.
Assessment: UnresolvedSaid
“It autocompletes sentences, some of those completions are true given that the training data is so vast, and some aren’t, given that the training data aren’t infinite.”
What we tested: GPT autocompletes sentences; some outputs are true because training data is vast and some are false because training data is not infinite.
Assessment: SupportedSaid
“It’s fun to compare the two—they have so many differences—even though neither can be trusted.”
What we tested: GPT cannot be trusted.
What happened: The preserved table reports hallucinations in 3.0% of GPT-4 summaries and 3.5% of GPT-3.5 Turbo summaries, despite providing source documents. These recurring errors support independent checking; the percentages apply to this summarization test, not all GPT answers. (Vectara, )
Held from this post (2)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The record does not meet the current rules. Our reading, not assessed: GPT's mistruths can look a lot like George Santos' lies, but that resemblance does not mean the machinery in Santos' brain has anything to do with GPT's GPUs.
- The two reviews reached different assessments. Our reading, not assessed: GPT has a truthiness problem that is not deliberate but arises because it doesn't know any better.