The first known chatbot associated death
We recorded 15 claims from this post. 5 were checked against evidence. 10 can’t be checked. 5 more are held. Read the post at its source.
Claims from this post (15)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: SupportedSaid
Observation
“The Chatbot involved was GPT-J, not GPT-3 or ChatGPT (contra some news headlines that were erroneous).”
Our reading: The chatbot involved in the Belgian suicide case was GPT-J, not GPT-3 or ChatGPT.
No horizon Evidence as of
Limitation: The evidence is reporting based on developer interviews, rather than an independently inspected deployment record.
Assessment: Can’t be checkedSaid
Causal claim
“But there’s some reason to think the chatbot may have contributed, as discussed below.”
Our reading: There is some reason to think the chatbot may have contributed to the Belgian man's suicide, though causality is murky.
No horizon
Qualifier: causality cannot be fully established
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Vulnerable patients shouldn’t be talking to chatbots that aren’t competent for this situation, and too much of the discussion in the media have made chatbots sound more intelligent and even more friendly than they are.”
Our reading: Vulnerable patients should not be talking to chatbots that are not competent for mental-health situations.
No horizon
Qualifier: chatbots that aren't competent for this situation
Assessment: UnresolvedSaid
Observation
“Vulnerable patients shouldn’t be talking to chatbots that aren’t competent for this situation, and too much of the discussion in the media have made chatbots sound more intelligent and even more friendly than they are.”
Our reading: Too much media discussion has made chatbots sound more intelligent and friendly than they are.
No horizon Evidence as of
Limitation: The phrase “too much” remains partly evaluative.
Assessment: SupportedSaid
Forecast
“Two reasons, first, because I believe that more incidents like this will happen, perhaps many more, hundreds or thousands, given how rapidly chatbots are being deployed, and the near total lack of regulation on how they can be used.”
Our reading: More chatbot-associated incidents like this suicide will happen, perhaps hundreds or thousands, given rapid deployment and near-total lack of regulation.
Settled before its deadline Evidence as of
Limitation: The filings do not establish causation or liability [e1, e2, e3].
Assessment: MixedSaid
Observation
“Two reasons, first, because I believe that more incidents like this will happen, perhaps many more, hundreds or thousands, given how rapidly chatbots are being deployed, and the near total lack of regulation on how they can be used.”
Our reading: There is a near-total lack of regulation on how chatbots can be used, and they are being deployed rapidly.
No horizon Evidence as of
Limitation: The result depends on reading regulation to include existing binding law.
Assessment: SupportedSaid
Observation
“The fundamental issue is that chatbots are neither reliable nor truthful, and at the same time don’t really grasp things like human psychology.)”
Our reading: Chatbots are neither reliable nor truthful and do not really grasp human psychology.
No horizon Evidence as of
Limitation: The result depends on the operational reading of really grasp.
Assessment: Can’t be checkedSaid
Forecast
“It took only a couple months for the first incident to pop up; we may see a lot more.”
Our reading: We may see many more chatbot-associated deaths/incidents.
No horizon
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“We should address this situation, even if there are others.”
Our reading: Society should address chatbot-associated mental-health harms even though other causes exist.
No horizon
Qualifier: even if other causes of suicide exist
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“No; it just means we probably shouldn’t use them to deliver therapy unless and until we become more confident in them.”
Our reading: We probably should not use LLMs to deliver therapy unless and until we become more confident in them.
No horizon
Qualifier: unless and until we become more confident in them
Assessment: Can’t be checkedSaid
Recommendation or value judgment
“Let’s use LLM’s where there can be help, and be cautious where they might cause harm.”
Our reading: LLMs should be used where they can help and with caution where they might cause harm.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“Ultimately I see the societal impact of those three as likely larger than the chatbot-administered medical and psychiatric advice, but I do remain quite concerned about the implications of not always competent medical and psychiatric advice.”
Our reading: The societal impact of LLM misinformation, cybercrime, and employment disruption will likely be larger than that of chatbot-administered medical and psychiatric advice.
No horizon
Assessment: Can’t be checkedSaid
Forecast
“I am particularly concerned that wholesale, plausible misinformation may create an atmosphere of mistrust, potentially undermining democracy; I also worry that large language models might also be weaponized to trick users as parts of large-scale (perhaps unprecedentedly large scale) cybercrime operations.”
Our reading: LLMs might be weaponized to trick users in large-scale, perhaps unprecedentedly large, cybercrime operations.
No horizon
Qualifier: near-term
Assessment: Can’t be checkedSaid
Forecast
“Medium-term we will likely see significant disruptions of employment.”
Our reading: In the medium term, we will likely see significant disruptions of employment.
No horizon
Scope: among potential harms from chatbots/large language models
Assessment: Can’t be checkedSaid
Observation
“The experience was tragic, and underscores the fact that vulnerable people may over attribute wisdom and intelligence to machines that don’t really understand humanity.”
Our reading: Chatbots do not really understand humanity, and vulnerable people may over-attribute wisdom and intelligence to them.
No horizon
Held from this post (5)
These records did not pass every check, so nothing about them is published. Held is not a finding.