Don’t Go Breaking My Heart
We recorded 13 claims from this post. 7 were checked against evidence. 6 can’t be checked. 3 more are held. Read the post at its source.
Claims from this post (13)
In the order they appear in the post. Each quotation is the speaker’s words in this post. A label such as “Supported” or “Can’t be checked” is the claim’s current assessment; its page has the evidence and the reasons.
Assessment: Can’t be checkedSaid
“the bigger large language models are, the more likely someone’s gonna get hurt.”
What we tested: In his prediction for 2023, the bigger large language models are, the more likely it is that someone will get hurt.
Assessment: SupportedSaid
“Sex and love, even simulated, are powerful urges; some people are apparently in genuine emotional pain, as result of the change.”
What we tested: Replika's removal of its erotic role-play feature has apparently caused some users genuine emotional pain.
What happened: ABC's interviews connect distress to the February change in intimate interactions. The article dates the changes to around Valentine's Day, while also identifying broader personality changes as a possible contributor. (Australian Broadcasting Corporation, )
Assessment: Can’t be checkedSaid
“This is a moment we should learn from. Hopefully nothing bad happens this time; but we need to reflect about what kind of society we are building.”
What we tested: Society should learn from the Replika episode and reflect on what kind of society it is building with chatbots.
Assessment: UnresolvedSaid
“Few people understand how they work; many people anthropomorphizing those chatbots, attributing to them real intelligence and emotion.”
What we tested: Few people understand how chatbots work, and many anthropomorphize them, attributing real intelligence and emotion to them.
Assessment: Can’t be checkedSaid
“Larger language models seems more and more human-like (but the emotions that they present are no more real).”
What we tested: Larger language models seem increasingly human-like, but the emotions they present are not real.
Assessment: Can’t be checkedSaid
“Whatever we see now is likely to escalate.”
What we tested: The current trend of increasingly human-like chatbots and its effects is likely to escalate.
Assessment: SupportedSaid
“Some people are building real attachments to those bots”
What we tested: Some people are forming real emotional attachments to chatbots.
What happened: The interview describes attachment and distress surrounding changes around Valentine’s Day. This is retrospective reporting about the relevant February experience. (Australian Broadcasting Corporation, )
Assessment: SupportedSaid
“In some cases, those who are building bots that actively cultivate those attachments, e.g., by feigning romantic and/or sexual interest or by dotting their messages with “friendly” emoticons.”
What we tested: Some chatbot builders design bots that actively cultivate user attachment, e.g. by feigning romantic or sexual interest.
What happened: The report describes paid flirting and erotic roleplay features present before the February removal. Offering these features supports the ordinary reading of inviting romantic attachment. (Australian Broadcasting Corporation, )
Assessment: Can’t be checkedSaid
“Changes in those bots could leave many people in a vulnerable place.”
What we tested: Changes to chatbots that users are attached to could leave many people emotionally vulnerable.
Assessment: UnresolvedSaid
“But no known technology can reliably align a chatbot in a persistent way to a human’s emotional needs.”
What we tested: No known technology can reliably and persistently align a chatbot to a human's emotional needs.
Assessment: ContradictedSaid
“To my knowledge, tech companies are free to leverage a human gullibility around chatbot technologies however they like, without consequence”
What we tested: To his knowledge, tech companies can exploit human gullibility about chatbots however they like without consequence.
What happened: This primary regulatory announcement reports an immediately effective restriction on Luka's processing of Italian users' data, citing transparency failures and risks to children and emotionally vulnerable people. It establishes a concrete constraint before Marcus's post, although it does not establish a general prohibition on simulated affection. (Garante per la protezione dei dati personali, )
Assessment: UnresolvedSaid
“with the new generation of chatbots, we will see addictions no less potent.”
What we tested: The new generation of chatbots will produce addictions at least as potent as social media addiction.
Assessment: Can’t be checkedSaid
“All this is one one more thing for Congress to take note of, as we start to consider policy in our Strange New World.”
What we tested: Congress should take chatbot attachment and manipulation risks into account when making AI policy.
Held from this post (3)
These records did not pass every check, so nothing about them is published. Held is not a finding.
- The criteria are ambiguous. Our reading, not assessed: There is essentially no regulation of what chatbots can say or do, how they change, or how they treat users.
- The two reviews read the claim differently
- The record does not meet the current rules. Our reading, not assessed: A leading chatbot (Replika) just changed its behavior radically for all users, possibly as a side effect of the Bing/Sydney fiasco.