AI literacy, hallucinations, and the law: A case study
We recorded 9 claims from this post. 4 were checked against evidence. 5 can’t be checked. 3 more are held. Read the post at its source.
Claims from this post (9)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
Rhetoric
“Straight up: the hypesters are winning.”
Our reading: AI hypesters are currently winning the public narrative.
No horizon
Assessment: SupportedSaid
Observation
“Take hallucinations, a deep, serious, unsolved problem with Generative AI, which I have been warning about since 2001.”
Our reading: Hallucination is a deep, serious, unsolved problem with generative AI.
No horizon Evidence as of
Limitation: Hallucination rates depend heavily on the benchmark and the model. The cited figures come from a few evaluations and do not give one general rate.
Assessment: SupportedSaid
Observation
“LLMs still can’t sanity check their own work or still purely to known sources, or notify when they have invented something bogus.”
Our reading: LLMs cannot sanity-check their own work, stick purely to known sources, or flag when they have invented something.
No horizon Evidence as of
Limitation: The result depends on reading 'can't' as 'can't reliably'. A strictly literal reading would fail, because models sometimes admit fabrication.
Assessment: SupportedSaid
Forecast
“Bullshit. Hallucinations are still here, and aren’t going away, anytime soon.”
Our reading: LLM hallucinations persist and will not go away anytime soon.
Deadline passed Evidence as of
Limitation: The one-year window is the evaluator's reading of 'anytime soon'.
Assessment: Can’t be checkedSaid
Observation
“And these are just the folks that got caught and that were publicized – probably a tiny fraction of the overall incidence.”
Our reading: Publicized cases of AI-fabricated legal citations are probably a tiny fraction of the total incidence.
No horizon
Assessment: UnresolvedSaid
Observation
“In all, Charlotin’s database lists 112 cases, for an average of a bit less than one publicly reported case a week since ChatGPT become popular. And if May is representative, the problem is getting worse.”
Our reading: Charlotin's database lists 112 cases, an average of a bit less than one publicly reported case a week since ChatGPT became popular, and if May is representative, the problem is getting worse.
No horizon Evidence as of
Only if: if May is representative
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“If educated people like lawyers still haven’t gotten the message that LLMs hallucinate, and for that matter that they can be fined or publicly humiliated if they rely on LLMs, something is fundamentally wrong.”
Our reading: If educated people like lawyers still haven't gotten the message that LLMs hallucinate, and that they can be fined or publicly humiliated if they rely on LLMs, something is fundamentally wrong.
No horizon
Only if: if educated people like lawyers still haven't gotten the message that LLMs hallucinate and that they can be fined or publicly humiliated for relying on LLMs
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“The media can and should do more to alert people to the shortcomings of these systems.”
Our reading: The media should do more to alert people to the shortcomings of LLMs, including how regularly hallucinations happen.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Those in the media who have downplayed hallucinations and oversold current AI have done the public a disservice.”
Our reading: Those in the media who have downplayed hallucinations and oversold current AI have done the public a disservice.
No horizon
Scope: those in the media who have downplayed hallucinations and oversold current AI
Held from this post (3)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The two reviews read the claim differently
- The two reviews reached different assessments. Our reading, not assessed: LLM hallucinated case citations in legal filings are a routine occurrence, not an anomaly.
- The criteria are ambiguous. Our reading, not assessed: A lot of lawyers are using tools like ChatGPT to prepare briefs, and many of them still seem shocked when it makes up cases.